Theory Wiki - User contributions [en]

Features of Adaptive Assistance that Improve Peer Tutoring in Algebra (Walker, Rummel, Koedinger)

2010-02-02T01:59:50Z

Erin-Walker:

== Collaborative Extensions to the Cognitive Tutor Algebra: Adaptive Assistance for Peer Tutoring ==
''Erin Walker, Nikol Rummel, and Ken Koedinger''

== Summary Tables ==
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''PI''' || Erin Walker
|-
| '''Co-PIs''' || Nikol Rummel, Ken Koedinger
|}
 

== Abstract ==
Adaptive collaborative learning support, where an intelligent system assesses student collaboration as it occurs and provides assistance when necessary, is a promising area of research. While fixed forms of support such as scripting student interaction have had a positive effect on collaboration quality, they can overconstrain the interaction for some students and provide too little help for others. Using intelligent tutoring technology to support collaboration might be more effective, but little is known about how to build these adaptive systems for collaboration and what effects they might have. We explore this area of research by augmenting an existing intelligent tutoring system with a peer tutoring activity and providing automated adaptive support to the activity.

This project has focused on how to improve the construction of adaptive collaboration systems with respect to their suitability for classroom deployment and the breadth of the models they employ. Most currently implemented systems are prototypes which are limited both in the scope of interaction that they support and in their use by students. In our first PSLC project, “[[Walker A Peer Tutoring Addition|Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition]]", we explored the advantages of refactoring an existing intelligent tutoring system in order to transform it into a platform for collaborative research, such that interface and tutoring components can be added and removed in order to create different research conditions. We next demonstrated how individual intelligent tutoring models could be used as input to collaboration models in order to better assess peer tutoring behaviors, in the PSLC project "[[Adaptive Assistance for Peer Tutoring (Walker, Rummer, Koedinger)|Collaborative Extensions to the Cognitive Tutor Algebra: Adaptive Assistance for Peer Tutoring]]". In this project, we extend this work by examining how individual models of student domain skills can be used as input to interaction models.

A related area of research is the potential of adaptive support for improving student interaction. The majority of the adaptive collaborative learning systems that have been developed have not been evaluated, and thus it is still unclear what influence adaptive support has compared to other forms of support. Our first step in this area was to develop adaptive domain support for the peer tutor, and compare it to a condition where the peer tutor is simply given problem solutions. While both types of support had advantages and disadvantages, it was clear peer tutors needed assistance that targeted collaboration skills in addition to domain knowledge. The next iteration of the system added adaptive interaction support to the adaptive domain support, and we compared the combined assistance to a fixed condition in a classroom study. As part of this project, we analyzed the study data in order to identify the broad impact both types of support have on the quality of student interaction, finding that adaptive support improves the quality of student help over fixed support. We now propose to investigate in more detail the potential role adaptive feedback could play in assisting student interaction by: 1) using HCI design methodologies to examine how students perceive and react to different features of support, and 2) empirically evaluating whether adaptive support has a cognitive or motivational influence on students. As an outcome of this research, we expect to add to understanding of the mechanisms by which adaptive support has an impact on student interaction, and how the support should be provided.

== Background & Significance ==

'''Peer Tutoring: Learning by Teaching'''

Incorporating peer tutoring into the CTA might be a way to encourage deep learning. Roscoe and Chi conclude that peer tutors benefit due to knowledge-building, where they reflect on their current knowledge and use it as a basis for constructing new knowledge (Roscoe & Chi, 2007). Because these positive effects are independent of tutor domain ability, researchers implement reciprocal peer tutoring programs, where students of similar abilities take turns tutoring each other. This type of peer tutoring has been shown to increase academic achievement and positive attitudes in long-term classroom interventions (Fantuzzo, Riggio, Connely, & Dimeff, 1989). Biswas et al. (2005) described three properties of peer tutoring related to tutor learning: tutors are accountable for their tutee’s knowledge, they reflect on tutee actions, and they engage in asking questions and giving explanations. Tutee learning is maximized at times when the tutee reaches an impasse, is prompted to find and explain the correct step, and is given an explanation if they fail to do so (VanLehn et al., 2003).

Peer tutors rarely exhibit knowledge-building behaviors spontaneously (Roscoe & Chi, 2007), and thus successful interventions provide them with assistance in order to achieve better learning outcomes for them and their tutees. This assistance can target tutoring behaviors through training, providing positive examples, or structuring the tutoring activities. For example, training students to give conceptual explanations had a significantly positive effect on learning (Fuchs et al., 1997). It is just as critical for assistance to target domain expertise of the peer tutors, in order to ensure that they have sufficient knowledge about a problem to help their partner solve it. Otherwise, there may be cognitive consequences (tutees cannot correctly solve problems) and affective consequences (students feel that they are poor tutors and become discouraged; Medway & Baron, 1997). Domain assistance can take the form of preparation on the problems and scaffolding during tutoring (e.g., Fantuzzo, Riggio, Connely, & Dimeff, 1989). Although assistance for peer tutoring has generally been fixed, providing adaptive support may be a promising approach.

'''Adaptive Collaborative Learning Systems'''

In order to benefit from collaboration students must interact in productive ways, and collaborative activities can be structured (scripted) to encourage these behaviors (e.g., Rummel & Spada, 2007). However, fixed scripts implemented in a one-size-fits-all fashion may be too restrictive for some students and place a high cognitive demand on others (Rummel & Spada, 2007; Dillenbourg, 2002). An adaptive system would be able to monitor student behaviors and provide support only when needed. Preliminary results suggest that adaptive support is indeed beneficial: Adaptive prompting realized in a Wizard of Oz fashion has been shown to have a positive effect on interaction and learning compared to an unscripted condition (Gweon, Rose, Carey, & Zaiss, 2006). An effective way to deliver this support would be to use an adaptive collaborative learning system, where feedback on collaboration is delivered by an intelligent agent.

Work on adaptive collaborative learning systems is still at an early stage. One approach is to use machine learning to detect problematic elements of student interaction in real-time and trigger helpful prompts. Although implementations have lead to significant learning gains, the adaptive feedback appears to be disruptive to dyadic interaction (Kumar et al., 2007). Another promising approach has explored using an intelligent agent as one of the collaborators; students teach the agent about ecosystems with the help of a mentoring agent (Biswas et al. 2005). However, the agents do not interact with the students in natural language, one of the primary benefits of collaboration. With respect to peer tutoring, intelligent tutoring technology could be applied either to supporting tutor behaviors or supporting the domain knowledge of peer tutors.

== Glossary ==
See [[:Category:Peer Tutoring|Peer Tutoring Glossary]]

== Research Questions ==

Can individual problem-solving models improve the effectiveness of adaptive collaborative learning support by providing more problem-solving context for models of collaboration? Do they make it easier to construct adaptive collaborative learning support systems?

What are the differential effects of adaptive and fixed support on student collaborative process during a peer tutoring activity, the acquisition of help-giving skills, and the resulting [[robust learning]] outcomes? Does adaptive support improve student ability to collaborate, student motivation to collaborate, or both?

== Independent Variables ==
1. ''Actual adaptivity of interaction support.'' We vary whether students are given support with highly relevant content at the moments they need it, or support with random content at moments when it is not needed.
2. ''Perceived adaptivity of interaction support.'' We vary whether students believe the support they are receiving is adaptively or randomly chosen.
3. ''Student role.'' We vary whether students take on the tutor or tutee role.

 Peer tutoring in the Cognitive Tutor Algebra. Adaptive interaction support received by the peer tutor. 
[[Image:Walker_adaptive__interaction_support.jpg]]

== Hypotheses ==
1. Peer tutors that show effective tutoring behaviors will show more domain learning than students that show ineffective tutoring behaviors.

2. Peer tutees that receive good tutoring will show more domain learning than peer tutees that receive bad tutoring.

3. Peer tutors that believe the assistance that they are receiving is adaptive will improve the quality of their tutoring, because they will feel more accountable for their behaviors.

4. Peer tutors that receive adaptive assistance will improve the quality of their tutoring, because they will be able to more easily apply the assistance to their behaviors.

== Dependent variables ==
* ''[[Normal post-test]]'': Students are given a brief post-test immediately after each study day on isomorphic problems
* ''Far [[transfer]]'': This paper and pencil test assessed students' understanding of the main mathematical concepts from the learning phase. The transfer items students had to solve tapped the same knowledge components as the problems in instruction, however, the problems where non-isomorphic to those in the instruction, thus demanded students to flexibly apply their knowledge to problems with a new format.
* ''Collaboration posttest'': Students collaborate without support in order to determine if they've improved their tutoring skills.

To compare [[collaboration skill]]s of students, we will be conducting an analysis of student dialogs during the learning phase.

To assess immediate effects of the instructional variations, we will analyze student progress on training problems as they work through the instruction.

== Findings ==
We are in the process of conducting a lab study with roughly 120 students. Out of this study, we expect to analyze in detail the effects of adaptive interaction support on student interaction, student acquisition of collaborative skills, and domain learning.

== Annotated bibliography ==
* Walker, E., Rummel, N., & Koedinger, K. R. Integrating collaboration and cognitive tutoring data in evaluation of a reciprocal peer tutoring environment. Research and Practice in Technology Enhanced Learning.
* Walker, E., Rummel, N., & Koedinger, K. R. CTRL: A Research Architecture for Providing Adaptive Collaborative Learning Support. User Modeling and User-Adapted Interaction.
* Walker, E., Rummel, N., and Koedinger, K. R. To Tutor the Tutor: Adaptive Domain Support for Peer Tutoring. To appear at the 9th International Conference on Intelligent Tutoring Systems. 2008.
* Walker, E., McLaren, B. M., Rummel, N., and Koedinger, K. R. Who Says Three's a Crowd? Using a Cognitive Tutor to Support Peer Tutoring. 13th International Conference on Artificial Intelligence and Education. 2007.
* Walker, E., Rummel, N., McLaren, B. M. & Koedinger, K. R. The Student Becomes the Master: Integrating Peer Tutoring with Cognitive Tutoring. Short paper at the Conference on Computer Supported Collaborative Learning (CSCL-07). Rutgers University, July 16-21, 2007.
* Walker, E., Koedinger, K., McLaren, B. M., & Rummel, N. (2006). Cognitive tutors as research platforms: Extending an established tutoring system for collaborative and metacognitive experimentation. ''Lecture Notes in Computer Science, Volume 4053/2006. Proceedings of the 8th International Conference on Intelligent Tutoring Systems'' (pp. 207-216). Berlin: Springer
* Walker, E. (2005). Mutual peer tutoring: A collaborative addition to the Algebra-1 Cognitive Tutor. Paper presented at the 12th International Conference on Artificial Intelligence and Education (AIED-05, Young Researchers Track), July, 2005, Amsterdam, the Netherlands.

== References ==
* Roscoe, R. D. & Chi, M. Understanding tutor learning: Knowledge-building and knowledge-telling in peer tutors’ explanations and questions. Review of Educational Research 77(4), 534-574 (2007)
* Fantuzzo, J. W., Riggio, R. E., Connelly, S., & Dimeff, L. A. Effects of reciprocal peer tutoring on academic achievement and psychological adjustment: A component analysis. Journal of Educational Psychology 81(2), 173-177 (1989)
* Biswas, G., Schwartz, D. L., Leelawong, K., Vye, N., & TAG-V. Learning by teaching: A new agent paradigm for educational software. Applied Artificial Intelligence 19, 363–392 (2005)
* VanLehn, K., Siler, S., Murray, C., Yamauchi, T., & Baggett, W. Why do only some events cause learning during human tutoring? Cognition and Instruction 21(3), 209-249 (2003)
* Fuchs, L., Fuchs, D., Hamlett, C., Phillips, N., Karns, K., & Dutka, S. Enhancing students’ helping behaviour during peer-mediated instruction with conceptual mathematical explanations. The Elementary School Journal 97(3), 223-249 (1997)
* Medway, F. & Baron, R. Locus of control and tutors’ instructional style. Contemporary Educational Psychology, 2, 298-310 (1997).
* Rummel, N. & Spada, H. Can people learn computer-mediated collaboration by following a script? In F. Fischer, I. Kollar, H. Mandl &, J. Haake, Scripting computer-supported communication of knowledge. Cognitive, computational, and educational perspectives (pp. 47-63). New York: Springer. (2007)
* Dillenbourg, P. Over-scripting CSCL: The risks of blending collaborative learning with instructional design. In P. A. Kirschner (Ed.), Three worlds of CSCL. Can we support CSCL (pp. 61-91). Heerlen: Open Universiteit Nederland. (2002)
* Gweon, G., Rosé, C., Carey, R. & Zaiss, Z. Providing Support for Adaptive Scripting in an On-Line Collaborative Learning Environment. Proc. of CHI 2006, pp. 251-260. (2006)
* Kumar, R., Rosé, C. P., Wang, Y. C., Joshi, M., Robinson, A. Tutorial dialogue as adaptive collaborative learning support. Proceedings of the 13th International Conference on Artificial Intelligence in Education (AIED 2007), Amsterdam: IOSPress. (2007)

== Connections ==
This study is an extension of the PSLC project "[[Walker A Peer Tutoring Addition|Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition]]" and "[[Adaptive Assistance for Peer Tutoring (Walker, Rummer, Koedinger)|Collaborative Extensions to the Cognitive Tutor Algebra: Adaptive Assistance for Peer Tutoring]]."

Like this study, [[Rummel Scripted Collaborative Problem Solving]] adds scripted collaborative problem solving to the Cognitive Tutor Algebra. The studies differ in the way collaboration is integrated in the Tutor. First, in the Rummel et al. study, both students first prepare one subtasks of a problem to mutually solve the complex story problem later on. Thus, although the students are experts for different parts of the problem, they have a comparable knowledge level during collaboration. In contrast, in this study, one student prepares to teach his partner. Then, they change roles. Thus, during collaboration, their knowledge level differs. Second, in the Rummel et al. study, collaboration was face to face, whereas this study used a chat tool for interaction.

Similar to the adaptive script component of the Collaborative Problem-Solving Script, the [[The Help Tutor Roll Aleven McLaren|Help Tutor project]] aims at improving students' [[help-seeking behavior]] and at reducing students' tendency to [[game the system]]. Furthermore, both studies contain instructions to teach [[metacognition]]. The metacognitive component in our study instructs students to monitor their interaction in order to improve it in subsequent collaborations; the Help Tutor project asks students to evaluate their need for help in order to improve their help-seeking behavior when learning on the Tutor.

Both in this study and in the [[Reflective Dialogues (Katz)|Reflective Dialogue study]] from Katz, students are asked to engage in reflection following each problem-solving. In this study, the reflection concentrates on the collaborative skills, while in Katz' study, the reflection concentrates on students' domain knowledge of the main principles applied in the problem.

Furthermore, both our study and the [[Help Lite (Aleven, Roll)]] aim at improving conceptual knowledge.

This project relates the more general thrust goals as follows. It is examining how features of assistance affect the three aspects of accountable talk: accountability to knowledge, accountability to rigorous thinking, and accountability to the learning community. Steps are being made toward the ambitious goal to operationalize and assess these aspects of accountable in real time as students interact and receive assistance in this computer-mediated environment. There is also a potential to code the three way dialog (student tutee, student tutor, and computer tutor) for transactivity. In particular, the student tutee's dialog moves have not yet been coded, but appear to have interesting elements, like asking for specific help or self-explaining, that may well connect to transactivity codes. Finally, there is a potential to analyze the computer tutor's reflective prompts for similarity with accountable talk moves and associated effectiveness. Some of the prompts were indeed inspired by accountable talk moves.

[[Category:Study]]

Social and Communicative Factors in Learning

2010-02-02T01:35:26Z

Erin-Walker: /* Descendants */

During PSLC’s first four years, its [[Interactive Communication]] Cluster has studied interactions between a student and a tutor (either human or computer) or, less frequently, two students interacting with each other. Most of the experimental manipulations and subsequent analyses have focused on the cognitive content of interaction through learning space analyses, in other words, the what and when of instruction. Study results investigating the effect of interaction, although somewhat mixed, have largely supported the hypothesis that focused interaction promotes cognitive aspects of learning such as attention to the most important knowledge components in a domain, deeper cognitive processing, and increased engagement with the content. VanLehn and colleagues (2007) present a thorough review of this literature as well as results from recent investigations. These results encouraged early PSLC efforts to “unpack” the nature of communicative interaction in instruction and learning. Rummel and colleagues (Diziol, Rummel, Kahrimanis, et al., 2008a, 2008b), for example have recently evaluated interactions with a rating scheme analysis that quantifies the quality of an interaction on a number of dimensions. This work represents an important step towards the type of up close inspection of communication that many scholars believe is necessary if we are to understand, and be able to manipulate for instructional purposes, how communication works to produce robust learning.

In our re-named Social-Communicative Factors thrust, we propose now to expand our investigations of communication as a core enabler of robust learning to include detailed study of patterns of interaction, the role of conversation and structured talk in initiating and sustaining learning, and the effects on motivation, self-attribution and commitment to a learning group that are associated with learning through social-communicative interaction. Specifically, we propose to investigate how human linguistic interaction works in instruction and learning, and how participants in learning exchanges (both teachers and students) can best be taught productive forms of interaction. We draw from our extensive prior work related separately to classroom discourse (Chapin & O’Connor, 2004; Bill et al., 1992; Resnick et al., 1992) and collaborative learning (Gweon et al., 2007; Joshi & Rosé, 2007; Rummel & Diziol, 2008). We note that, although the classroom discourse and collaborative learning communities have proceeded mainly independently from one another, the conversational processes identified as valuable within these two communities are strongly overlapping.

Investigations of valuable conversational contributions have been conducted both within communities exploring the cognitive foundations of group learning and the sociocultural community. Regardless of the theoretical framework, the same ideas have surfaced under a number of different names including [[Accountable Talk]] (Michaels, O’Connor & Resnick, 2007; Resnick, O'Connor, & Michaels, 2007), transactivity (Berkowitz & Gibbs, 1984; Teasley, 1997; Weinberger & Fishcer, 2006; King, 1999), productive agency (Schwartz, 1999), and uptake (Suthers, 2006), and have been demonstrated to predict learning both in collaborative learning contexts (Azimita & Montgomery, 1993; Joshi & Rosé, 2007) and classroom contexts (O’Connor et al., 2007). For example, one cognitive justification for the value of transactive conversational behavior is its connection with cognitive conflict (Piaget, 1985), where transactive conversational moves highlight differences between the mental models of collaborating students. One can argue that a major cognitive benefit of collaborative learning is that when students bring differing perspectives to a problem-solving situation, the interaction causes the participants to consider questions that might not have occurred to them otherwise. This stimulus could cause them to identify gaps in their understanding, which they would then be in a position to address. This type of cognitive conflict has the potential to lead to productive shifts in student understanding. It has the potential to elicit elaborate explanations from students that are associated with learning (Webb, Nemer, & Zuniga 2002). From the sociocultural perspective, based on Vygotsky’s seminal work (Vygotsky 1978), we can similarly argue that when students who have different strengths and weaknesses work together, they can provide support for each other that allows them to solve problems that would be just beyond their reach if they were working alone.

We will proceed with two interacting research strategies: one, expanding capacities for recording, coding and analyzing interactive communication that can be at least partially automated; and two, conducting in vivo experiments on ways of teaching participants the most promising patterns of interactive communication and testing the effects of these patterns on measures of robust learning.

In the first thread of our proposed work, we will work toward a common conceptual framework that unifies the classroom discourse, collaborative learning and instructional tutoring communities. To this end, we plan to develop a concrete and precise formalization on a linguistic level of what counts as performing these valued conversational moves. This concrete formalization will provide a common language for documenting and investigating the specific ways in which social-communicative practices can promote (or hinder) learning of complex mathematics and science content and reasoning skills.

In the second thread of our proposed work, we will examine causal connections between these communicative processes and learning by running in vivo experiments in which specific social-communicative practices are introduced into well-defined mathematics and science units of study. We will begin by replicating and extending a series of in vivo experiments on the effects of [[Accountable Talk]] in low-income urban classrooms with high proportions of English language learners in Chelsea, Massachusetts (O’Connor et al 2007; NHSF REC 0231893, PI: O’Connor). In a tightly controlled series of three-day studies in 5th and 6th grade classrooms, O’Connor’s group sought to determine whether it was possible to get evidence supporting a hypothesized causal relationship between selected discourse-intensive instructional practices and student mathematics learning. In previous non-experimental studies in Chelsea, students had shown large gains on standardized tests after a year or more of discourse-intensive instruction, but it was not possible to test the specific features of the intervention that produced these effects. Thus it was possible that cognitive and metacognitive abilities might improve over months of practice in clarifying, justifying and describing mathematical ideas, whether or not explicit transactive communication strategies were employed. Similarly, student motivation might have improved due to long-term participation in an intensive mathematics program, without a specific impact of particular forms of linguistic participation.

We will design and run in vivo experiments to test more specific hypotheses concerning specific [[Accountable Talk]] moves. Subsequent studies will test a larger intervention that includes training in the most effective conversational moves and collaborative scripts with implementation in a number of classrooms. The studies will focus on math and science learning topics. These studies will make use of techniques from automatic collaborative learning process analysis (Rosé et al., in press; Wang et al., 2007; Donmez et al., 2005) and script-based support for productive collaboration (Dillenbourg & Jermann, 2007; Kollar, Fischer, & Hesse, 2006; Rummel & Spada, 2007; Diziol, Rummel, Kahrimanis, Spada & Avaris, 2008; Diziol et al., 2008; Walker, Rummel, McLaren & Koedinger, 2007) to carefully manipulate these properties of conversation in highly controlled and context sensitive ways.

== Descendants ==

To create a new project page, enclose your project name in a double set of brackets. Details for a project format may be [[ Project_Page_Template_and_Creation_Instructions | found here.]]

*[[Rose - Integrated framework for analysis of classroom discussions]]
*[[Features of Adaptive Assistance that Improve Peer Tutoring in Algebra (Walker, Rummel, Koedinger)]]
*[[Resnick Project]]

== References ==
* Walker, E., Rummel, N., & Koedinger, K. (2008). A Research-Oriented Architecture for Providing Adaptive Collaborative Learning Support [[Media: Walker_Architecture_for_Learing.pdf‎ | Click to download]]

* Chi, M.T., Roy, M., & Hausmann, R.G. (March, 2008). Observing tutorial dialogues collaboratively: Insights about human tutoring effectiveness from vicarious learning. Cognitive Science: A Multidisciplinary Journal, 32:2, 301-341. [[Media:Chi_Observing_Tutorial_Dialogues.pdf | Click to download]]

* Meier, A., Spada, H. & Rummel, N. (2007). A rating scheme for assessing the quality of computer-supported collaboration processes. International Journal of Computer-Supported Collaborative Learning, 2, 63-86. [[Media: Meier_Rating_Scheme.pdf| Click to download]]

* Resnick, L., O'Connor, C., and Michaels, S. (2007). Classroom Discourse, Mathematical Rigor, and Student Reasoning: An Accountable Talk Literature Review.[[Media: Accountable_Talk_Lit_Review.pdf | Click to download]]

* Rose, C., et al. (2007). Analyzing collaborative learning processes automatically: Exploiting the advance of computational linguistics in computer-supported collaborative learning. [[Media: Rose_Analyzing_Collaborative.pdf | Click to download]]

* Yamakawa,Y., Forman, E., and Ansell, E. (2005). The role of positioning in constructing an identity in a third grade mathematics classroom. [[Media: Yamakawa_role_of_positioning.pdf| Click to download]]

Features of Adaptive Assistance that Improve Peer Tutoring in Algebra (Walker, Rummel, Koedinger)

2009-12-11T22:45:05Z

Erin-Walker:

Features of Adaptive Assistance that Improve Peer Tutoring in Algebra (Walker, Rummel, Koedinger)

2009-12-11T22:36:48Z

Erin-Walker:

Features of Adaptive Assistance that Improve Peer Tutoring in Algebra (Walker, Rummel, Koedinger)

2009-12-11T22:27:31Z

Erin-Walker:

Features of Adaptive Assistance that Improve Peer Tutoring in Algebra (Walker, Rummel, Koedinger)

2009-12-11T22:24:41Z

Erin-Walker: /* Abstract */

Features of Adaptive Assistance that Improve Peer Tutoring in Algebra (Walker, Rummel, Koedinger)

2009-12-11T22:22:15Z

Erin-Walker: /* Abstract */

Features of Adaptive Assistance that Improve Peer Tutoring in Algebra (Walker, Rummel, Koedinger)

2009-12-05T23:38:10Z

Erin-Walker: New page: == Collaborative Extensions to the Cognitive Tutor Algebra: Adaptive Assistance for Peer Tutoring == ''Erin Walker, Nikol Rummel, and Ken Koedinger'' == Summary Tables == {| border="1" c...

Adaptive Assistance for Peer Tutoring (Walker, Rummer, Koedinger)

2009-05-18T05:09:27Z

Erin-Walker: /* Findings */

== Collaborative Extensions to the Cognitive Tutor Algebra: Adaptive Assistance for Peer Tutoring ==
''Erin Walker, Nikol Rummel, and Ken Koedinger''

=== Summary Tables ===
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''PI''' || Erin Walker
|-
| '''Co-PIs''' || Nikol Rummel, Ken Koedinger
|}
 

=== Abstract ==
Adaptive collaborative learning support, where an intelligent system assesses student collaboration as it occurs and provides assistance when necessary, is a promising area of research. While fixed forms of support such as scripting student interaction have had a positive effect on collaboration quality, they can overconstrain the interaction for some students and provide too little help for others. Using intelligent tutoring technology to support collaboration might be more effective, but little is known about how to build these adaptive systems for collaboration and what effects they might have. We explore this area of research by augmenting an existing intelligent tutoring system with a peer tutoring activity and providing automated adaptive support to the activity.

This project has focused on how to improve the construction of adaptive collaboration systems with respect to their suitability for classroom deployment and the breadth of the models they employ. Most currently implemented systems are prototypes which are limited both in the scope of interaction that they support and in their use by students. In a previous PSLC project, “[[Walker A Peer Tutoring Addition|Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition]], we explored the advantages of refactoring an existing intelligent tutoring system in order to transform it into a platform for collaborative research, such that interface and tutoring components can be added and removed in order to create different research conditions. We next demonstrated how individual intelligent tutoring models could be used as input to collaboration models in order to better assess peer tutoring behaviors. We propose to further extend this work by examining how individual models of student domain skills can be used as input to interaction models, and how collaborative skills can be represented.
A related area of research is the potential of adaptive support for improving student interaction. The majority of the adaptive collaborative learning systems that have been developed have not been evaluated, and thus it is still unclear what influence adaptive support has compared to other form of support. Our first step in this area was to develop adaptive domain support for the peer tutor, and compare it to a condition where the peer tutor is simply given problem solutions. While both types of support had advantages and disadvantages, it was clear peer tutors needed assistance that targeted collaboration skills in addition to domain knowledge. The next iteration of the system added adaptive interaction support to the adaptive domain support, and currently we am comparing the combined assistance to a fixed condition in a classroom study. Our first proposed step will be to analyze the study data in order to identify the broad impact both types of support have on the quality of student interaction. We will then investigate in more detail the potential role adaptive feedback could play in assisting student interaction by: 1) using HCI design methodologies to examine how students perceive and react to different features of support, and 2) empirically evaluating how the directness of different support presentations influences student behavior. As an outcome of this research, we expect to add to understanding of the mechanisms by which adaptive support has an impact on student interaction, and how the support should be provided.

=== Background and Significance ===

'''Peer Tutoring: Learning by Teaching'''

Incorporating peer tutoring into the CTA might be a way to encourage deep learning. Roscoe and Chi conclude that peer tutors benefit due to knowledge-building, where they reflect on their current knowledge and use it as a basis for constructing new knowledge (Roscoe & Chi, 2007). Because these positive effects are independent of tutor domain ability, researchers implement reciprocal peer tutoring programs, where students of similar abilities take turns tutoring each other. This type of peer tutoring has been shown to increase academic achievement and positive attitudes in long-term classroom interventions (Fantuzzo, Riggio, Connely, & Dimeff, 1989). Biswas et al. (2005) described three properties of peer tutoring related to tutor learning: tutors are accountable for their tutee’s knowledge, they reflect on tutee actions, and they engage in asking questions and giving explanations. Tutee learning is maximized at times when the tutee reaches an impasse, is prompted to find and explain the correct step, and is given an explanation if they fail to do so (VanLehn et al., 2003).

Peer tutors rarely exhibit knowledge-building behaviors spontaneously (Roscoe & Chi, 2007), and thus successful interventions provide them with assistance in order to achieve better learning outcomes for them and their tutees. This assistance can target tutoring behaviors through training, providing positive examples, or structuring the tutoring activities. For example, training students to give conceptual explanations had a significantly positive effect on learning (Fuchs et al., 1997). It is just as critical for assistance to target domain expertise of the peer tutors, in order to ensure that they have sufficient knowledge about a problem to help their partner solve it. Otherwise, there may be cognitive consequences (tutees cannot correctly solve problems) and affective consequences (students feel that they are poor tutors and become discouraged; Medway & Baron, 1997). Domain assistance can take the form of preparation on the problems and scaffolding during tutoring (e.g., Fantuzzo, Riggio, Connely, & Dimeff, 1989). Although assistance for peer tutoring has generally been fixed, providing adaptive support may be a promising approach.

'''Adaptive Collaborative Learning Systems'''

In order to benefit from collaboration students must interact in productive ways, and collaborative activities can be structured (scripted) to encourage these behaviors (e.g., Rummel & Spada, 2007). However, fixed scripts implemented in a one-size-fits-all fashion may be too restrictive for some students and place a high cognitive demand on others (Rummel & Spada, 2007; Dillenbourg, 2002). An adaptive system would be able to monitor student behaviors and provide support only when needed. Preliminary results suggest that adaptive support is indeed beneficial: Adaptive prompting realized in a Wizard of Oz fashion has been shown to have a positive effect on interaction and learning compared to an unscripted condition (Gweon, Rose, Carey, & Zaiss, 2006). An effective way to deliver this support would be to use an adaptive collaborative learning system, where feedback on collaboration is delivered by an intelligent agent.

Work on adaptive collaborative learning systems is still at an early stage. One approach is to use machine learning to detect problematic elements of student interaction in real-time and trigger helpful prompts. Although implementations have lead to significant learning gains, the adaptive feedback appears to be disruptive to dyadic interaction (Kumar et al., 2007). Another promising approach has explored using an intelligent agent as one of the collaborators; students teach the agent about ecosystems with the help of a mentoring agent (Biswas et al. 2005). However, the agents do not interact with the students in natural language, one of the primary benefits of collaboration.

With respect to peer tutoring, intelligent tutoring technology could be applied either to supporting tutor behaviors or domain knowledge of peer tutors. As it is very difficult to build an intelligent tutor for collaborative processes, we decided to develop a general script for the peer tutoring interaction and then focus on providing adaptive domain assistance to peer tutors by leveraging the existing domain models of the CTA. A condition where students tutor each other with adaptive domain support provided to the peer tutor is likely to be better than a condition where the peer tutor merely has access to an answer key, because the support would be tailored to each individual tutor’s needs. It is also likely to be better than a condition where students use the CTA individually, because the students in the collaborative condition would be able to interact deeply about the domain material.

=== Glossary ===
See [[:Category:Peer Tutoring|Peer Tutoring Glossary]]

=== Research Question ===

Can individual problem-solving models improve the effectiveness of adaptive collaborative learning support by providing more problem-solving context for models of collaboration? Do they make it easier to construct adaptive collaborative learning support systems?

What are the differential effects of adaptive and fixed support on student collaborative process during a peer tutoring activity, the acquisition of help-giving skills, and the resulting [[robust learning]] outcomes? How do the effects vary with different types of support?

=== Independent variables ===
We vary the [[agent]]s involved in the interaction using the [[instructional method|mode of instruction]]. For example, students may interact individually with a [[cognitive tutor]], may interact with each other using a [[collaboration script]], or may interact with each other with the help of a [[cognitive tutor]].

In a series of studies, we further examine the type of assistance provided to the students. They may receive assistance on how to solve the problem, what good collaboration is, and how to collaborate well.

Finally, we vary within subjects whether students take on the tutor or tutee role.

Currently, we vary whether students collaborate or work individually, and the adaptivity of the support they receive:

Figure 1. Individual learning in the Cognitive Tutor Algebra. 
[[Image:Walker_individual_learning.gif]]

Figure 2. Peer tutoring in the Cognitive Tutor Algebra. Peer tutor's interface. 
[[Image:Walker_peer_tutoring.jpg]]

 Figure 3. Peer tutoring in the Cognitive Tutor Algebra. Adaptive domain support received by the peer tutor. 
[[Image:Walker_adaptive_support.jpg]]

 Figure 4. Peer tutoring in the Cognitive Tutor Algebra. Adaptive interaction support received by the peer tutor. 
[[Image:Walker_adaptive__interaction_support.jpg]]

=== Hypotheses ===
1. Students that show high tutoring competence behaviors will show
more domain learning than students that show low tutoring competence behaviors

2. Either domain or collaboration assistance alone is better than no assistance at promoting good peer tutoring behaviors. Providing peer tutors with
both domain and collaboration assistance will lead them to show better peer
tutoring behaviors than with only domain or collaboration assistance.

4. An increase in positive peer tutoring behaviors due to fixed
assistance will lead to an increase in peer tutor robust domain learning

5. An increase in positive peer tutoring behaviors due to fixed assistance
will lead to an increase in peer tutee robust domain learning

6. Adaptive assistance is more effective than fixed assistance at
improving peer tutoring behaviors, which promotes robust domain learning of the
peer tutor and peer tutee.

7. Students who receive fixed assistance will learn more from being tutors, while students who receive adaptive assistance will learn equally from both roles.

=== Dependent variables ===
* ''[[Normal post-test]]'': Students are given a brief post-test immediately after each study day on isomorphic problems
* ''Far [[transfer]]'': This paper and pencil test assessed students' understanding of the main mathematical concepts from the learning phase. The transfer items students had to solve tapped the same knowledge components as the problems in instruction, however, the problems where non-isomorphic to those in the instruction, thus demanded students to flexibly apply their knowledge to problems with a new format.
* ''[[Long-term retention]]'': Students will be given a test a month after the immediate posttest.
* ''[[Accelerated future learning]] test'': Student learning on future equation solving units will be measured.

To compare [[collaboration skill]]s of students, we will be conducting an analysis of student dialogs during the learning phase.

To assess immediate effects of the instructional variations, we will analyze student progress on training problems as they work through the instruction.

=== Findings ===
We are in the process of conducting a classroom study in two schools with roughly 90 students in which we vary whether students receive adaptive or fixed assistance and whether they assume the tutor or tutee role first. Data collection is ongoing.

Out of this study, we expect to analyze in detail the effects of adaptive domain and interaction support on student interaction, student acquisition of collaborative skills, and domain learning.

=== Annotated bibliography ===
* Walker, E., Rummel, N., & Koedinger, K. R. Integrating collaboration and cognitive tutoring data in evaluation of a reciprocal peer tutoring environment. Submitted to Research and Practice in Technology Enhanced Learning. Accepted with minor revisions.
* Walker, E., Rummel, N., & Koedinger, K. R. CTRL: A Research Architecture for Providing Adaptive Collaborative Learning Support. Submitted to User Modeling
and User-Adapted Interaction. Accepted with minor revisions.
* Walker, E., Rummel, N., and Koedinger, K. R. To Tutor the Tutor: Adaptive Domain Support for Peer Tutoring. To appear at the 9th International Conference on Intelligent Tutoring Systems. 2008.
* Walker, E., McLaren, B. M., Rummel, N., and Koedinger, K. R. Who Says Three's a Crowd? Using a Cognitive Tutor to Support Peer Tutoring. 13th International Conference on Artificial Intelligence and Education. 2007.
* Walker, E., Rummel, N., McLaren, B. M. & Koedinger, K. R. The Student Becomes the Master: Integrating Peer Tutoring with Cognitive Tutoring. Short paper at the Conference on Computer Supported Collaborative Learning (CSCL-07). Rutgers University, July 16-21, 2007.
* Walker, E., Koedinger, K., McLaren, B. M., & Rummel, N. (2006). Cognitive tutors as research platforms: Extending an established tutoring system for collaborative and metacognitive experimentation. ''Lecture Notes in Computer Science, Volume 4053/2006. Proceedings of the 8th International Conference on Intelligent Tutoring Systems'' (pp. 207-216). Berlin: Springer
* Walker, E. (2005). Mutual peer tutoring: A collaborative addition to the Algebra-1 Cognitive Tutor. Paper presented at the 12th International Conference on Artificial Intelligence and Education (AIED-05, Young Researchers Track), July, 2005, Amsterdam, the Netherlands.

=== References ===
* Roscoe, R. D. & Chi, M. Understanding tutor learning: Knowledge-building and knowledge-telling in peer tutors’ explanations and questions. Review of Educational Research 77(4), 534-574 (2007)
* Fantuzzo, J. W., Riggio, R. E., Connelly, S., & Dimeff, L. A. Effects of reciprocal peer tutoring on academic achievement and psychological adjustment: A component analysis. Journal of Educational Psychology 81(2), 173-177 (1989)
* Biswas, G., Schwartz, D. L., Leelawong, K., Vye, N., & TAG-V. Learning by teaching: A new agent paradigm for educational software. Applied Artificial Intelligence 19, 363–392 (2005)
* VanLehn, K., Siler, S., Murray, C., Yamauchi, T., & Baggett, W. Why do only some events cause learning during human tutoring? Cognition and Instruction 21(3), 209-249 (2003)
* Fuchs, L., Fuchs, D., Hamlett, C., Phillips, N., Karns, K., & Dutka, S. Enhancing students’ helping behaviour during peer-mediated instruction with conceptual mathematical explanations. The Elementary School Journal 97(3), 223-249 (1997)
* Medway, F. & Baron, R. Locus of control and tutors’ instructional style. Contemporary Educational Psychology, 2, 298-310 (1997).
* Rummel, N. & Spada, H. Can people learn computer-mediated collaboration by following a script? In F. Fischer, I. Kollar, H. Mandl &, J. Haake, Scripting computer-supported communication of knowledge. Cognitive, computational, and educational perspectives (pp. 47-63). New York: Springer. (2007)
* Dillenbourg, P. Over-scripting CSCL: The risks of blending collaborative learning with instructional design. In P. A. Kirschner (Ed.), Three worlds of CSCL. Can we support CSCL (pp. 61-91). Heerlen: Open Universiteit Nederland. (2002)
* Gweon, G., Rosé, C., Carey, R. & Zaiss, Z. Providing Support for Adaptive Scripting in an On-Line Collaborative Learning Environment. Proc. of CHI 2006, pp. 251-260. (2006)
* Kumar, R., Rosé, C. P., Wang, Y. C., Joshi, M., Robinson, A. Tutorial dialogue as adaptive collaborative learning support. Proceedings of the 13th International Conference on Artificial Intelligence in Education (AIED 2007), Amsterdam: IOSPress. (2007)

=== Connections ===
This study is an extension of the PSLC project "[[Walker A Peer Tutoring Addition|Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition]]."

Like this study, [[Rummel Scripted Collaborative Problem Solving]] adds scripted collaborative problem solving to the Cognitive Tutor Algebra. The studies differ in the way collaboration is integrated in the Tutor. First, in the Rummel et al. study, both students first prepare one subtasks of a problem to mutually solve the complex story problem later on. Thus, although the students are experts for different parts of the problem, they have a comparable knowledge level during collaboration. In contrast, in this study, one student prepares to teach his partner. Then, they change roles. Thus, during collaboration, their knowledge level differs. Second, in the Rummel et al. study, collaboration was face to face, whereas this study used a chat tool for interaction.

Similar to the adaptive script component of the Collaborative Problem-Solving Script, the [[The Help Tutor Roll Aleven McLaren|Help Tutor project]] aims at improving students' [[help-seeking behavior]] and at reducing students' tendency to [[game the system]]. Furthermore, both studies contain instructions to teach [[metacognition]]. The metacognitive component in our study instructs students to monitor their interaction in order to improve it in subsequent collaborations; the Help Tutor project asks students to evaluate their need for help in order to improve their help-seeking behavior when learning on the Tutor.

Both in this study and in the [[Reflective Dialogues (Katz)|Reflective Dialogue study]] from Katz, students are asked to engage in reflection following each problem-solving. In this study, the reflection concentrates on the collaborative skills, while in Katz' study, the reflection concentrates on students' domain knowledge of the main principles applied in the problem.

Furthermore, both our study and the [[Help Lite (Aleven, Roll)]] aim at improving conceptual knowledge.

=== Future plans ===
Our future plans for June 2009 - September 2009:
* Analyze the study data

[[Category:Study]]

File:Walker adaptive interaction support.jpg

2009-05-18T05:05:17Z

Erin-Walker:

Adaptive Assistance for Peer Tutoring (Walker, Rummer, Koedinger)

2009-05-18T05:01:22Z

Erin-Walker: /* = Abstract */

== Collaborative Extensions to the Cognitive Tutor Algebra: Adaptive Assistance for Peer Tutoring ==
''Erin Walker, Nikol Rummel, and Ken Koedinger''

=== Summary Tables ===
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''PI''' || Erin Walker
|-
| '''Co-PIs''' || Nikol Rummel, Ken Koedinger
|}
 

=== Abstract ==
Adaptive collaborative learning support, where an intelligent system assesses student collaboration as it occurs and provides assistance when necessary, is a promising area of research. While fixed forms of support such as scripting student interaction have had a positive effect on collaboration quality, they can overconstrain the interaction for some students and provide too little help for others. Using intelligent tutoring technology to support collaboration might be more effective, but little is known about how to build these adaptive systems for collaboration and what effects they might have. We explore this area of research by augmenting an existing intelligent tutoring system with a peer tutoring activity and providing automated adaptive support to the activity.

This project has focused on how to improve the construction of adaptive collaboration systems with respect to their suitability for classroom deployment and the breadth of the models they employ. Most currently implemented systems are prototypes which are limited both in the scope of interaction that they support and in their use by students. In a previous PSLC project, “[[Walker A Peer Tutoring Addition|Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition]], we explored the advantages of refactoring an existing intelligent tutoring system in order to transform it into a platform for collaborative research, such that interface and tutoring components can be added and removed in order to create different research conditions. We next demonstrated how individual intelligent tutoring models could be used as input to collaboration models in order to better assess peer tutoring behaviors. We propose to further extend this work by examining how individual models of student domain skills can be used as input to interaction models, and how collaborative skills can be represented.
A related area of research is the potential of adaptive support for improving student interaction. The majority of the adaptive collaborative learning systems that have been developed have not been evaluated, and thus it is still unclear what influence adaptive support has compared to other form of support. Our first step in this area was to develop adaptive domain support for the peer tutor, and compare it to a condition where the peer tutor is simply given problem solutions. While both types of support had advantages and disadvantages, it was clear peer tutors needed assistance that targeted collaboration skills in addition to domain knowledge. The next iteration of the system added adaptive interaction support to the adaptive domain support, and currently we am comparing the combined assistance to a fixed condition in a classroom study. Our first proposed step will be to analyze the study data in order to identify the broad impact both types of support have on the quality of student interaction. We will then investigate in more detail the potential role adaptive feedback could play in assisting student interaction by: 1) using HCI design methodologies to examine how students perceive and react to different features of support, and 2) empirically evaluating how the directness of different support presentations influences student behavior. As an outcome of this research, we expect to add to understanding of the mechanisms by which adaptive support has an impact on student interaction, and how the support should be provided.

=== Background and Significance ===

'''Peer Tutoring: Learning by Teaching'''

Incorporating peer tutoring into the CTA might be a way to encourage deep learning. Roscoe and Chi conclude that peer tutors benefit due to knowledge-building, where they reflect on their current knowledge and use it as a basis for constructing new knowledge (Roscoe & Chi, 2007). Because these positive effects are independent of tutor domain ability, researchers implement reciprocal peer tutoring programs, where students of similar abilities take turns tutoring each other. This type of peer tutoring has been shown to increase academic achievement and positive attitudes in long-term classroom interventions (Fantuzzo, Riggio, Connely, & Dimeff, 1989). Biswas et al. (2005) described three properties of peer tutoring related to tutor learning: tutors are accountable for their tutee’s knowledge, they reflect on tutee actions, and they engage in asking questions and giving explanations. Tutee learning is maximized at times when the tutee reaches an impasse, is prompted to find and explain the correct step, and is given an explanation if they fail to do so (VanLehn et al., 2003).

Peer tutors rarely exhibit knowledge-building behaviors spontaneously (Roscoe & Chi, 2007), and thus successful interventions provide them with assistance in order to achieve better learning outcomes for them and their tutees. This assistance can target tutoring behaviors through training, providing positive examples, or structuring the tutoring activities. For example, training students to give conceptual explanations had a significantly positive effect on learning (Fuchs et al., 1997). It is just as critical for assistance to target domain expertise of the peer tutors, in order to ensure that they have sufficient knowledge about a problem to help their partner solve it. Otherwise, there may be cognitive consequences (tutees cannot correctly solve problems) and affective consequences (students feel that they are poor tutors and become discouraged; Medway & Baron, 1997). Domain assistance can take the form of preparation on the problems and scaffolding during tutoring (e.g., Fantuzzo, Riggio, Connely, & Dimeff, 1989). Although assistance for peer tutoring has generally been fixed, providing adaptive support may be a promising approach.

'''Adaptive Collaborative Learning Systems'''

In order to benefit from collaboration students must interact in productive ways, and collaborative activities can be structured (scripted) to encourage these behaviors (e.g., Rummel & Spada, 2007). However, fixed scripts implemented in a one-size-fits-all fashion may be too restrictive for some students and place a high cognitive demand on others (Rummel & Spada, 2007; Dillenbourg, 2002). An adaptive system would be able to monitor student behaviors and provide support only when needed. Preliminary results suggest that adaptive support is indeed beneficial: Adaptive prompting realized in a Wizard of Oz fashion has been shown to have a positive effect on interaction and learning compared to an unscripted condition (Gweon, Rose, Carey, & Zaiss, 2006). An effective way to deliver this support would be to use an adaptive collaborative learning system, where feedback on collaboration is delivered by an intelligent agent.

Work on adaptive collaborative learning systems is still at an early stage. One approach is to use machine learning to detect problematic elements of student interaction in real-time and trigger helpful prompts. Although implementations have lead to significant learning gains, the adaptive feedback appears to be disruptive to dyadic interaction (Kumar et al., 2007). Another promising approach has explored using an intelligent agent as one of the collaborators; students teach the agent about ecosystems with the help of a mentoring agent (Biswas et al. 2005). However, the agents do not interact with the students in natural language, one of the primary benefits of collaboration.

With respect to peer tutoring, intelligent tutoring technology could be applied either to supporting tutor behaviors or domain knowledge of peer tutors. As it is very difficult to build an intelligent tutor for collaborative processes, we decided to develop a general script for the peer tutoring interaction and then focus on providing adaptive domain assistance to peer tutors by leveraging the existing domain models of the CTA. A condition where students tutor each other with adaptive domain support provided to the peer tutor is likely to be better than a condition where the peer tutor merely has access to an answer key, because the support would be tailored to each individual tutor’s needs. It is also likely to be better than a condition where students use the CTA individually, because the students in the collaborative condition would be able to interact deeply about the domain material.

=== Glossary ===
See [[:Category:Peer Tutoring|Peer Tutoring Glossary]]

=== Research Question ===

Can individual problem-solving models improve the effectiveness of adaptive collaborative learning support by providing more problem-solving context for models of collaboration? Do they make it easier to construct adaptive collaborative learning support systems?

What are the differential effects of adaptive and fixed support on student collaborative process during a peer tutoring activity, the acquisition of help-giving skills, and the resulting [[robust learning]] outcomes? How do the effects vary with different types of support?

=== Independent variables ===
We vary the [[agent]]s involved in the interaction using the [[instructional method|mode of instruction]]. For example, students may interact individually with a [[cognitive tutor]], may interact with each other using a [[collaboration script]], or may interact with each other with the help of a [[cognitive tutor]].

In a series of studies, we further examine the type of assistance provided to the students. They may receive assistance on how to solve the problem, what good collaboration is, and how to collaborate well.

Finally, we vary within subjects whether students take on the tutor or tutee role.

Currently, we vary whether students collaborate or work individually, and the adaptivity of the support they receive:

Figure 1. Individual learning in the Cognitive Tutor Algebra. 
[[Image:Walker_individual_learning.gif]]

Figure 2. Peer tutoring in the Cognitive Tutor Algebra. Peer tutor's interface. 
[[Image:Walker_peer_tutoring.jpg]]

 Figure 3. Peer tutoring in the Cognitive Tutor Algebra. Adaptive domain support received by the peer tutor. 
[[Image:Walker_adaptive_support.jpg]]

 Figure 4. Peer tutoring in the Cognitive Tutor Algebra. Adaptive interaction support received by the peer tutor. 
[[Image:Walker_adaptive__interaction_support.jpg]]

=== Hypotheses ===
1. Students that show high tutoring competence behaviors will show
more domain learning than students that show low tutoring competence behaviors

2. Either domain or collaboration assistance alone is better than no assistance at promoting good peer tutoring behaviors. Providing peer tutors with
both domain and collaboration assistance will lead them to show better peer
tutoring behaviors than with only domain or collaboration assistance.

4. An increase in positive peer tutoring behaviors due to fixed
assistance will lead to an increase in peer tutor robust domain learning

5. An increase in positive peer tutoring behaviors due to fixed assistance
will lead to an increase in peer tutee robust domain learning

6. Adaptive assistance is more effective than fixed assistance at
improving peer tutoring behaviors, which promotes robust domain learning of the
peer tutor and peer tutee.

7. Students who receive fixed assistance will learn more from being tutors, while students who receive adaptive assistance will learn equally from both roles.

=== Dependent variables ===
* ''[[Normal post-test]]'': Students are given a brief post-test immediately after each study day on isomorphic problems
* ''Far [[transfer]]'': This paper and pencil test assessed students' understanding of the main mathematical concepts from the learning phase. The transfer items students had to solve tapped the same knowledge components as the problems in instruction, however, the problems where non-isomorphic to those in the instruction, thus demanded students to flexibly apply their knowledge to problems with a new format.
* ''[[Long-term retention]]'': Students will be given a test a month after the immediate posttest.
* ''[[Accelerated future learning]] test'': Student learning on future equation solving units will be measured.

To compare [[collaboration skill]]s of students, we will be conducting an analysis of student dialogs during the learning phase.

To assess immediate effects of the instructional variations, we will analyze student progress on training problems as they work through the instruction.

=== Findings ===
We are in the process of conducting a classroom study in two schools with roughly 90 students in which we vary whether students receive adaptive or fixed assistance and whether they assume the tutor or tutee role first. Data collection is ongoing.

=== Annotated bibliography ===
* Walker, E., Rummel, N., & Koedinger, K. R. Integrating collaboration and cognitive tutoring data in evaluation of a reciprocal peer tutoring environment. Submitted to Research and Practice in Technology Enhanced Learning. Accepted with minor revisions.
* Walker, E., Rummel, N., & Koedinger, K. R. CTRL: A Research Architecture for Providing Adaptive Collaborative Learning Support. Submitted to User Modeling
and User-Adapted Interaction. Accepted with minor revisions.
* Walker, E., Rummel, N., and Koedinger, K. R. To Tutor the Tutor: Adaptive Domain Support for Peer Tutoring. To appear at the 9th International Conference on Intelligent Tutoring Systems. 2008.
* Walker, E., McLaren, B. M., Rummel, N., and Koedinger, K. R. Who Says Three's a Crowd? Using a Cognitive Tutor to Support Peer Tutoring. 13th International Conference on Artificial Intelligence and Education. 2007.
* Walker, E., Rummel, N., McLaren, B. M. & Koedinger, K. R. The Student Becomes the Master: Integrating Peer Tutoring with Cognitive Tutoring. Short paper at the Conference on Computer Supported Collaborative Learning (CSCL-07). Rutgers University, July 16-21, 2007.
* Walker, E., Koedinger, K., McLaren, B. M., & Rummel, N. (2006). Cognitive tutors as research platforms: Extending an established tutoring system for collaborative and metacognitive experimentation. ''Lecture Notes in Computer Science, Volume 4053/2006. Proceedings of the 8th International Conference on Intelligent Tutoring Systems'' (pp. 207-216). Berlin: Springer
* Walker, E. (2005). Mutual peer tutoring: A collaborative addition to the Algebra-1 Cognitive Tutor. Paper presented at the 12th International Conference on Artificial Intelligence and Education (AIED-05, Young Researchers Track), July, 2005, Amsterdam, the Netherlands.

=== References ===
* Roscoe, R. D. & Chi, M. Understanding tutor learning: Knowledge-building and knowledge-telling in peer tutors’ explanations and questions. Review of Educational Research 77(4), 534-574 (2007)
* Fantuzzo, J. W., Riggio, R. E., Connelly, S., & Dimeff, L. A. Effects of reciprocal peer tutoring on academic achievement and psychological adjustment: A component analysis. Journal of Educational Psychology 81(2), 173-177 (1989)
* Biswas, G., Schwartz, D. L., Leelawong, K., Vye, N., & TAG-V. Learning by teaching: A new agent paradigm for educational software. Applied Artificial Intelligence 19, 363–392 (2005)
* VanLehn, K., Siler, S., Murray, C., Yamauchi, T., & Baggett, W. Why do only some events cause learning during human tutoring? Cognition and Instruction 21(3), 209-249 (2003)
* Fuchs, L., Fuchs, D., Hamlett, C., Phillips, N., Karns, K., & Dutka, S. Enhancing students’ helping behaviour during peer-mediated instruction with conceptual mathematical explanations. The Elementary School Journal 97(3), 223-249 (1997)
* Medway, F. & Baron, R. Locus of control and tutors’ instructional style. Contemporary Educational Psychology, 2, 298-310 (1997).
* Rummel, N. & Spada, H. Can people learn computer-mediated collaboration by following a script? In F. Fischer, I. Kollar, H. Mandl &, J. Haake, Scripting computer-supported communication of knowledge. Cognitive, computational, and educational perspectives (pp. 47-63). New York: Springer. (2007)
* Dillenbourg, P. Over-scripting CSCL: The risks of blending collaborative learning with instructional design. In P. A. Kirschner (Ed.), Three worlds of CSCL. Can we support CSCL (pp. 61-91). Heerlen: Open Universiteit Nederland. (2002)
* Gweon, G., Rosé, C., Carey, R. & Zaiss, Z. Providing Support for Adaptive Scripting in an On-Line Collaborative Learning Environment. Proc. of CHI 2006, pp. 251-260. (2006)
* Kumar, R., Rosé, C. P., Wang, Y. C., Joshi, M., Robinson, A. Tutorial dialogue as adaptive collaborative learning support. Proceedings of the 13th International Conference on Artificial Intelligence in Education (AIED 2007), Amsterdam: IOSPress. (2007)

=== Connections ===
This study is an extension of the PSLC project "[[Walker A Peer Tutoring Addition|Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition]]."

Like this study, [[Rummel Scripted Collaborative Problem Solving]] adds scripted collaborative problem solving to the Cognitive Tutor Algebra. The studies differ in the way collaboration is integrated in the Tutor. First, in the Rummel et al. study, both students first prepare one subtasks of a problem to mutually solve the complex story problem later on. Thus, although the students are experts for different parts of the problem, they have a comparable knowledge level during collaboration. In contrast, in this study, one student prepares to teach his partner. Then, they change roles. Thus, during collaboration, their knowledge level differs. Second, in the Rummel et al. study, collaboration was face to face, whereas this study used a chat tool for interaction.

Similar to the adaptive script component of the Collaborative Problem-Solving Script, the [[The Help Tutor Roll Aleven McLaren|Help Tutor project]] aims at improving students' [[help-seeking behavior]] and at reducing students' tendency to [[game the system]]. Furthermore, both studies contain instructions to teach [[metacognition]]. The metacognitive component in our study instructs students to monitor their interaction in order to improve it in subsequent collaborations; the Help Tutor project asks students to evaluate their need for help in order to improve their help-seeking behavior when learning on the Tutor.

Both in this study and in the [[Reflective Dialogues (Katz)|Reflective Dialogue study]] from Katz, students are asked to engage in reflection following each problem-solving. In this study, the reflection concentrates on the collaborative skills, while in Katz' study, the reflection concentrates on students' domain knowledge of the main principles applied in the problem.

Furthermore, both our study and the [[Help Lite (Aleven, Roll)]] aim at improving conceptual knowledge.

=== Future plans ===
Our future plans for June 2009 - September 2009:
* Analyze the study data

[[Category:Study]]

Adaptive Assistance for Peer Tutoring (Walker, Rummer, Koedinger)

2009-05-18T05:00:56Z

Erin-Walker:

== Collaborative Extensions to the Cognitive Tutor Algebra: Adaptive Assistance for Peer Tutoring ==
''Erin Walker, Nikol Rummel, and Ken Koedinger''

=== Summary Tables ===
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''PI''' || Erin Walker
|-
| '''Co-PIs''' || Nikol Rummel, Ken Koedinger
|}
 

=== Abstract ==http://www.learnlab.org/research/wiki/index.php?title=Adaptive_Assistance_for_Peer_Tutoring_%28Walker%2C_Rummer%2C_Koedinger%29&action=edit=
Adaptive collaborative learning support, where an intelligent system assesses student collaboration as it occurs and provides assistance when necessary, is a promising area of research. While fixed forms of support such as scripting student interaction have had a positive effect on collaboration quality, they can overconstrain the interaction for some students and provide too little help for others. Using intelligent tutoring technology to support collaboration might be more effective, but little is known about how to build these adaptive systems for collaboration and what effects they might have. We explore this area of research by augmenting an existing intelligent tutoring system with a peer tutoring activity and providing automated adaptive support to the activity.

This project has focused on how to improve the construction of adaptive collaboration systems with respect to their suitability for classroom deployment and the breadth of the models they employ. Most currently implemented systems are prototypes which are limited both in the scope of interaction that they support and in their use by students. In a previous PSLC project, “[[Walker A Peer Tutoring Addition|Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition]], we explored the advantages of refactoring an existing intelligent tutoring system in order to transform it into a platform for collaborative research, such that interface and tutoring components can be added and removed in order to create different research conditions. We next demonstrated how individual intelligent tutoring models could be used as input to collaboration models in order to better assess peer tutoring behaviors. We propose to further extend this work by examining how individual models of student domain skills can be used as input to interaction models, and how collaborative skills can be represented.
A related area of research is the potential of adaptive support for improving student interaction. The majority of the adaptive collaborative learning systems that have been developed have not been evaluated, and thus it is still unclear what influence adaptive support has compared to other form of support. Our first step in this area was to develop adaptive domain support for the peer tutor, and compare it to a condition where the peer tutor is simply given problem solutions. While both types of support had advantages and disadvantages, it was clear peer tutors needed assistance that targeted collaboration skills in addition to domain knowledge. The next iteration of the system added adaptive interaction support to the adaptive domain support, and currently we am comparing the combined assistance to a fixed condition in a classroom study. Our first proposed step will be to analyze the study data in order to identify the broad impact both types of support have on the quality of student interaction. We will then investigate in more detail the potential role adaptive feedback could play in assisting student interaction by: 1) using HCI design methodologies to examine how students perceive and react to different features of support, and 2) empirically evaluating how the directness of different support presentations influences student behavior. As an outcome of this research, we expect to add to understanding of the mechanisms by which adaptive support has an impact on student interaction, and how the support should be provided.

=== Background and Significance ===

'''Peer Tutoring: Learning by Teaching'''

Incorporating peer tutoring into the CTA might be a way to encourage deep learning. Roscoe and Chi conclude that peer tutors benefit due to knowledge-building, where they reflect on their current knowledge and use it as a basis for constructing new knowledge (Roscoe & Chi, 2007). Because these positive effects are independent of tutor domain ability, researchers implement reciprocal peer tutoring programs, where students of similar abilities take turns tutoring each other. This type of peer tutoring has been shown to increase academic achievement and positive attitudes in long-term classroom interventions (Fantuzzo, Riggio, Connely, & Dimeff, 1989). Biswas et al. (2005) described three properties of peer tutoring related to tutor learning: tutors are accountable for their tutee’s knowledge, they reflect on tutee actions, and they engage in asking questions and giving explanations. Tutee learning is maximized at times when the tutee reaches an impasse, is prompted to find and explain the correct step, and is given an explanation if they fail to do so (VanLehn et al., 2003).

Peer tutors rarely exhibit knowledge-building behaviors spontaneously (Roscoe & Chi, 2007), and thus successful interventions provide them with assistance in order to achieve better learning outcomes for them and their tutees. This assistance can target tutoring behaviors through training, providing positive examples, or structuring the tutoring activities. For example, training students to give conceptual explanations had a significantly positive effect on learning (Fuchs et al., 1997). It is just as critical for assistance to target domain expertise of the peer tutors, in order to ensure that they have sufficient knowledge about a problem to help their partner solve it. Otherwise, there may be cognitive consequences (tutees cannot correctly solve problems) and affective consequences (students feel that they are poor tutors and become discouraged; Medway & Baron, 1997). Domain assistance can take the form of preparation on the problems and scaffolding during tutoring (e.g., Fantuzzo, Riggio, Connely, & Dimeff, 1989). Although assistance for peer tutoring has generally been fixed, providing adaptive support may be a promising approach.

'''Adaptive Collaborative Learning Systems'''

In order to benefit from collaboration students must interact in productive ways, and collaborative activities can be structured (scripted) to encourage these behaviors (e.g., Rummel & Spada, 2007). However, fixed scripts implemented in a one-size-fits-all fashion may be too restrictive for some students and place a high cognitive demand on others (Rummel & Spada, 2007; Dillenbourg, 2002). An adaptive system would be able to monitor student behaviors and provide support only when needed. Preliminary results suggest that adaptive support is indeed beneficial: Adaptive prompting realized in a Wizard of Oz fashion has been shown to have a positive effect on interaction and learning compared to an unscripted condition (Gweon, Rose, Carey, & Zaiss, 2006). An effective way to deliver this support would be to use an adaptive collaborative learning system, where feedback on collaboration is delivered by an intelligent agent.

Work on adaptive collaborative learning systems is still at an early stage. One approach is to use machine learning to detect problematic elements of student interaction in real-time and trigger helpful prompts. Although implementations have lead to significant learning gains, the adaptive feedback appears to be disruptive to dyadic interaction (Kumar et al., 2007). Another promising approach has explored using an intelligent agent as one of the collaborators; students teach the agent about ecosystems with the help of a mentoring agent (Biswas et al. 2005). However, the agents do not interact with the students in natural language, one of the primary benefits of collaboration.

With respect to peer tutoring, intelligent tutoring technology could be applied either to supporting tutor behaviors or domain knowledge of peer tutors. As it is very difficult to build an intelligent tutor for collaborative processes, we decided to develop a general script for the peer tutoring interaction and then focus on providing adaptive domain assistance to peer tutors by leveraging the existing domain models of the CTA. A condition where students tutor each other with adaptive domain support provided to the peer tutor is likely to be better than a condition where the peer tutor merely has access to an answer key, because the support would be tailored to each individual tutor’s needs. It is also likely to be better than a condition where students use the CTA individually, because the students in the collaborative condition would be able to interact deeply about the domain material.

=== Glossary ===
See [[:Category:Peer Tutoring|Peer Tutoring Glossary]]

=== Research Question ===

Can individual problem-solving models improve the effectiveness of adaptive collaborative learning support by providing more problem-solving context for models of collaboration? Do they make it easier to construct adaptive collaborative learning support systems?

What are the differential effects of adaptive and fixed support on student collaborative process during a peer tutoring activity, the acquisition of help-giving skills, and the resulting [[robust learning]] outcomes? How do the effects vary with different types of support?

=== Independent variables ===
We vary the [[agent]]s involved in the interaction using the [[instructional method|mode of instruction]]. For example, students may interact individually with a [[cognitive tutor]], may interact with each other using a [[collaboration script]], or may interact with each other with the help of a [[cognitive tutor]].

In a series of studies, we further examine the type of assistance provided to the students. They may receive assistance on how to solve the problem, what good collaboration is, and how to collaborate well.

Finally, we vary within subjects whether students take on the tutor or tutee role.

Currently, we vary whether students collaborate or work individually, and the adaptivity of the support they receive:

Figure 1. Individual learning in the Cognitive Tutor Algebra. 
[[Image:Walker_individual_learning.gif]]

Figure 2. Peer tutoring in the Cognitive Tutor Algebra. Peer tutor's interface. 
[[Image:Walker_peer_tutoring.jpg]]

 Figure 3. Peer tutoring in the Cognitive Tutor Algebra. Adaptive domain support received by the peer tutor. 
[[Image:Walker_adaptive_support.jpg]]

 Figure 4. Peer tutoring in the Cognitive Tutor Algebra. Adaptive interaction support received by the peer tutor. 
[[Image:Walker_adaptive__interaction_support.jpg]]

=== Hypotheses ===
1. Students that show high tutoring competence behaviors will show
more domain learning than students that show low tutoring competence behaviors

2. Either domain or collaboration assistance alone is better than no assistance at promoting good peer tutoring behaviors. Providing peer tutors with
both domain and collaboration assistance will lead them to show better peer
tutoring behaviors than with only domain or collaboration assistance.

4. An increase in positive peer tutoring behaviors due to fixed
assistance will lead to an increase in peer tutor robust domain learning

5. An increase in positive peer tutoring behaviors due to fixed assistance
will lead to an increase in peer tutee robust domain learning

6. Adaptive assistance is more effective than fixed assistance at
improving peer tutoring behaviors, which promotes robust domain learning of the
peer tutor and peer tutee.

7. Students who receive fixed assistance will learn more from being tutors, while students who receive adaptive assistance will learn equally from both roles.

=== Dependent variables ===
* ''[[Normal post-test]]'': Students are given a brief post-test immediately after each study day on isomorphic problems
* ''Far [[transfer]]'': This paper and pencil test assessed students' understanding of the main mathematical concepts from the learning phase. The transfer items students had to solve tapped the same knowledge components as the problems in instruction, however, the problems where non-isomorphic to those in the instruction, thus demanded students to flexibly apply their knowledge to problems with a new format.
* ''[[Long-term retention]]'': Students will be given a test a month after the immediate posttest.
* ''[[Accelerated future learning]] test'': Student learning on future equation solving units will be measured.

To compare [[collaboration skill]]s of students, we will be conducting an analysis of student dialogs during the learning phase.

To assess immediate effects of the instructional variations, we will analyze student progress on training problems as they work through the instruction.

=== Findings ===
We are in the process of conducting a classroom study in two schools with roughly 90 students in which we vary whether students receive adaptive or fixed assistance and whether they assume the tutor or tutee role first. Data collection is ongoing.

=== Annotated bibliography ===
* Walker, E., Rummel, N., & Koedinger, K. R. Integrating collaboration and cognitive tutoring data in evaluation of a reciprocal peer tutoring environment. Submitted to Research and Practice in Technology Enhanced Learning. Accepted with minor revisions.
* Walker, E., Rummel, N., & Koedinger, K. R. CTRL: A Research Architecture for Providing Adaptive Collaborative Learning Support. Submitted to User Modeling
and User-Adapted Interaction. Accepted with minor revisions.
* Walker, E., Rummel, N., and Koedinger, K. R. To Tutor the Tutor: Adaptive Domain Support for Peer Tutoring. To appear at the 9th International Conference on Intelligent Tutoring Systems. 2008.
* Walker, E., McLaren, B. M., Rummel, N., and Koedinger, K. R. Who Says Three's a Crowd? Using a Cognitive Tutor to Support Peer Tutoring. 13th International Conference on Artificial Intelligence and Education. 2007.
* Walker, E., Rummel, N., McLaren, B. M. & Koedinger, K. R. The Student Becomes the Master: Integrating Peer Tutoring with Cognitive Tutoring. Short paper at the Conference on Computer Supported Collaborative Learning (CSCL-07). Rutgers University, July 16-21, 2007.
* Walker, E., Koedinger, K., McLaren, B. M., & Rummel, N. (2006). Cognitive tutors as research platforms: Extending an established tutoring system for collaborative and metacognitive experimentation. ''Lecture Notes in Computer Science, Volume 4053/2006. Proceedings of the 8th International Conference on Intelligent Tutoring Systems'' (pp. 207-216). Berlin: Springer
* Walker, E. (2005). Mutual peer tutoring: A collaborative addition to the Algebra-1 Cognitive Tutor. Paper presented at the 12th International Conference on Artificial Intelligence and Education (AIED-05, Young Researchers Track), July, 2005, Amsterdam, the Netherlands.

=== References ===
* Roscoe, R. D. & Chi, M. Understanding tutor learning: Knowledge-building and knowledge-telling in peer tutors’ explanations and questions. Review of Educational Research 77(4), 534-574 (2007)
* Fantuzzo, J. W., Riggio, R. E., Connelly, S., & Dimeff, L. A. Effects of reciprocal peer tutoring on academic achievement and psychological adjustment: A component analysis. Journal of Educational Psychology 81(2), 173-177 (1989)
* Biswas, G., Schwartz, D. L., Leelawong, K., Vye, N., & TAG-V. Learning by teaching: A new agent paradigm for educational software. Applied Artificial Intelligence 19, 363–392 (2005)
* VanLehn, K., Siler, S., Murray, C., Yamauchi, T., & Baggett, W. Why do only some events cause learning during human tutoring? Cognition and Instruction 21(3), 209-249 (2003)
* Fuchs, L., Fuchs, D., Hamlett, C., Phillips, N., Karns, K., & Dutka, S. Enhancing students’ helping behaviour during peer-mediated instruction with conceptual mathematical explanations. The Elementary School Journal 97(3), 223-249 (1997)
* Medway, F. & Baron, R. Locus of control and tutors’ instructional style. Contemporary Educational Psychology, 2, 298-310 (1997).
* Rummel, N. & Spada, H. Can people learn computer-mediated collaboration by following a script? In F. Fischer, I. Kollar, H. Mandl &, J. Haake, Scripting computer-supported communication of knowledge. Cognitive, computational, and educational perspectives (pp. 47-63). New York: Springer. (2007)
* Dillenbourg, P. Over-scripting CSCL: The risks of blending collaborative learning with instructional design. In P. A. Kirschner (Ed.), Three worlds of CSCL. Can we support CSCL (pp. 61-91). Heerlen: Open Universiteit Nederland. (2002)
* Gweon, G., Rosé, C., Carey, R. & Zaiss, Z. Providing Support for Adaptive Scripting in an On-Line Collaborative Learning Environment. Proc. of CHI 2006, pp. 251-260. (2006)
* Kumar, R., Rosé, C. P., Wang, Y. C., Joshi, M., Robinson, A. Tutorial dialogue as adaptive collaborative learning support. Proceedings of the 13th International Conference on Artificial Intelligence in Education (AIED 2007), Amsterdam: IOSPress. (2007)

=== Connections ===
This study is an extension of the PSLC project "[[Walker A Peer Tutoring Addition|Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition]]."

Like this study, [[Rummel Scripted Collaborative Problem Solving]] adds scripted collaborative problem solving to the Cognitive Tutor Algebra. The studies differ in the way collaboration is integrated in the Tutor. First, in the Rummel et al. study, both students first prepare one subtasks of a problem to mutually solve the complex story problem later on. Thus, although the students are experts for different parts of the problem, they have a comparable knowledge level during collaboration. In contrast, in this study, one student prepares to teach his partner. Then, they change roles. Thus, during collaboration, their knowledge level differs. Second, in the Rummel et al. study, collaboration was face to face, whereas this study used a chat tool for interaction.

Similar to the adaptive script component of the Collaborative Problem-Solving Script, the [[The Help Tutor Roll Aleven McLaren|Help Tutor project]] aims at improving students' [[help-seeking behavior]] and at reducing students' tendency to [[game the system]]. Furthermore, both studies contain instructions to teach [[metacognition]]. The metacognitive component in our study instructs students to monitor their interaction in order to improve it in subsequent collaborations; the Help Tutor project asks students to evaluate their need for help in order to improve their help-seeking behavior when learning on the Tutor.

Both in this study and in the [[Reflective Dialogues (Katz)|Reflective Dialogue study]] from Katz, students are asked to engage in reflection following each problem-solving. In this study, the reflection concentrates on the collaborative skills, while in Katz' study, the reflection concentrates on students' domain knowledge of the main principles applied in the problem.

Furthermore, both our study and the [[Help Lite (Aleven, Roll)]] aim at improving conceptual knowledge.

=== Future plans ===
Our future plans for June 2009 - September 2009:
* Analyze the study data

[[Category:Study]]

Walker A Peer Tutoring Addition

2008-12-09T01:43:56Z

Erin-Walker: /* Independent variables */

== Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition ==
''Erin Walker, Bruce M. McLaren, Nikol Rummel, and Ken Koedinger''

=== Summary Tables ===
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''PIs''' || Bruce M. McLaren, Nikol Rummel, Ken Koedinger
|-
| '''Other Contributers''' ||
* Graduate Student: Erin Walker
* Staff: Jonathan Steinhart
|}
 
'' Pre Study 1''
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''Study Start Date''' || 05/06
|-
| '''Study End Date''' || 06/06
|-
| '''LearnLab Site''' || CWCTC
|-
| '''LearnLab Course''' || Algebra
|-
| '''Number of Students''' || ''N'' = 14
|-
| '''Average # of hours per participant''' || 3 hrs.
|}
 

'' Pre Study 2''
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''Study Start Date''' || 11/06
|-
| '''Study End Date''' || 11/06
|-
| '''LearnLab Site''' || CWCTC
|-
| '''LearnLab Course''' || Algebra
|-
| '''Number of Students''' || ''N'' = 20
|-
| '''Average # of hours per participant''' || 3 hrs.
|}
 

'' Full Study''
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''Study Start Date''' || 04/07
|-
| '''Study End Date''' || 04/07
|-
| '''LearnLab Site''' || CWCTC
|-
| '''LearnLab Course''' || Algebra
|-
| '''Number of Students''' || ''N'' = 70
|-
| '''Average # of hours per participant''' || 3 hrs.
|}
 

=== Abstract ===
In this project, the Algebra I Cognitive Tutor is extended to a [[collaborative learning environment]]: Instead of the computer tutoring a student, students tutor each other, and the [[cognitive tutor]] provides collaborative support. As research has shown, collaborative problem solving and learning has the potential to increase elaboration on the learning content. However, students are not always able to effectively meet the challenges of a collaborative setting. To ensure that students capitalize on collaborative problem solving with the Tutor, a [[peer tutoring]] [[collaboration scripts|collaboration script]] was developed that guides their interaction and prompts fruitful [[collaboration]].

In the baseline version of the script (without cognitive tutor support), students in the same class are seated at different computers and take turns tutoring each other. First, they prepare to tutor by solving problems using the regular cognitive tutor. Then, they tutor each other by marking each other right and wrong, monitoring their partner's skills, and give each other hints and feedback. After preparing and tutoring, they reflect on the problems and their interaction, engaging in [[metacognition|metacognitive activities]]. We have expanded the script to include intelligent cognitive support of the peer tutoring, and hope to ultimately expand it to include intelligent collaborative support.

To assess the effectiveness of the script on [[robust learning]], we are conducting a series of [[in vivo experiment]]s in the Algebra LearnLab. In an initial, small scale study (pre study) that served to establish basic effects and to test the procedure in a classroom setting, we compared peer tutoring with reflection elements to peer tutoring without reflection elements. In a second study that served to explore the effects of cognitive tutoring of peer tutoring, we compared peer tutoring plus cognitive tutoring to peer tutoring. In the full study, we will compare these collaborative conditions to an individual condition to assess the effect of the collaborative Tutor extension to regular Tutor use.

=== Background and Significance ===

In our project, we combined two different [[instructional method]]s both of which have been shown to improve students’ learning in mathematics: Learning with intelligent tutoring systems (Koedinger, Anderson, Hadley, & Mark, 1997) and collaborative problem solving (Berg, 1993). The [[Cognitive tutor|Cognitive Tutor]] Algebra that was used in our study is a tutor for mathematics instruction at the high school level. Students learn with the Tutor during part of the regular classroom sessions. The Tutor's main features are immediate error feedback, the possibility to ask for a hint when encountering impasses, and knowledge tracing, i.e. the Tutor creates and updates a model of the student’s knowledge and selects new problems tailored to the student’s knowledge level. Although several studies have proven its effectiveness, students do not always benefit from learning with the Tutor. First, because the Tutor places emphasis on learning [[procedural|procedural problem solving skills]], yet a deep understanding of underlying mathematical [[conceptual knowledge|concepts]] is not necessarily achieved (Anderson, Corbett, Koedinger, & Pelletier, 1995). Second, students do not always make good use of the learning opportunities provided by the Cognitive Tutor (e.g. [[help abuse]], see Aleven, McLaren, Roll, & Koedinger, 2004; [[game the system|gaming the system]], Baker, Corbett, & Koedinger, 2004). So far, the Cognitive Tutor has been used in an individual learning setting only. However, as research on collaborative learning has shown, collaboration can yield elaboration of learning content (Teasley, 1995), thus this could be a promising approach to reduce the Tutor’s shortcomings. On the other hand, students are not always able to effectively meet the challenges of a collaborative setting (Rummel & Spada, 2005). [[Collaboration scripts]] have proven effective in helping people meet the challenges encountered when learning or working collaboratively (Kollar, Fischer, & Hesse, in press).

Combining collaborative activities with intelligent tutoring might combine the benefits of both approaches. Collaborative interaction could augment the effects of an existing ITS by adding deeper interaction, and intelligent tutoring support could improve the quality of student collaboration by providing guidance to students as they attempt to follow a collaboration script. One way to incorporate collaboration into an ITS is to place the intelligent tutoring system in the role of one of the collaborators and have a student interact with this agent. For example, Biswas, Schwartz, Leelawong, Vye, and the TAG-V (2005) have the student collaborate with an agent (“Betty”) in a peer tutoring scenario. The student teaches Betty about ecosystems with the help of a second mentoring agent (“Mr. Davis”), and it has been shown that a student who takes the role of a teacher in this system benefits more than a student who takes the role of a learner in a parallel traditional tutoring system. However, this system does not yet support natural language interaction between the agents involved, and therefore does not take full advantage of important collaborative interaction mechanisms that can increase learning. Another way to combine collaboration and tutoring is to use the ITS to structure and tutor the student collaboration, by connecting collaborative tools with ITS components (see Harrer, McLaren, Walker, Bollen, & Sewall, 2006), and developing a model of good and ineffective collaboration in order to provide on-line hints and feedback (see Soller, 2001). Most research into computer tutoring of the collaboration between two students is still in a preliminary stage.

We integrate collaborative learning with an ITS using a peer tutoring framework. Similar to Biswas et al. (2005), we put the student in the role of the tutor, and intend to have the cognitive elements of the tutoring supported by another agent. However, unlike Biswas et al., (2005), the teachable agent is a real student. Therefore, we have intelligent tutoring components monitoring and providing feedback on student collaboration in addition to student cognition, like Harrer et al. (2006). Our ultimate goal is to allow students to tutor each other through the interface of an ITS, supported by both cognitive and collaborative tutoring.

=== Glossary ===
See [[:Category:Peer Tutoring|Peer Tutoring Glossary]]

=== Research question ===
Does [[collaboration]] – and in particular [[collaboration scripts|scripted collaboration]] – improve students’ [[robust learning]] in the domain of algebra?

Do [[instructional method|instructional activities]] that involve three agents (a student, a peer tutor, and a computer tutor) increase [[robust learning]] compared to instructional activities that involve two agents (student and peer tutor, student and computer tutor)?

=== Independent variables ===
We vary the [[agent]]s involved in the interaction using the [[instructional method|mode of instruction]]. There are four conditions:
* Individual condition (Figure 1): Students solved problems with the Algebra I Cognitive Tutor in the regular fashion (i.e. computer program as additional [[agent]]).
* Peer tutoring condition (Figure 2): Students tutored each other using the cognitive tutor interface (i.e., another peer as additional [[agent]]). They went through two phases:
** a preparation phase, where students prepared to tutor while solving problems with the Algebra I Cognitive Tutor in the regular fashion
** a collaboration phase, where students take turns tutoring each other. Peer tutors can mark answers right or wrong, monitor their partner's skills, and give hints and feedback
* Peer tutoring plus reflection: As in the peer tutoring condition, students go through a preparation phase and then tutor each other in the collaboration phase. However, students also go through a series of [[metacognition|metacognitive]] self-reflection activities to encourage them to engage in [[elaborative interaction]]
* Peer tutoring plus cognitive tutoring (Figure 3): Students participate in the peer tutoring plus reflection condition, but during the collaboration phase, peer tutors receive cognitive tutor hints and feedback (i.e., a peer and computer program as additional [[agent]]s). This feedback contains two components:
** a collaborative prompt that encourages students to interact with each other
** a cognitive hint targeted at the particular mistake the peer tutee is making

 Figure 1. Individual learning in the Cognitive Tutor Algebra. 
[[Image:Walker_individual_learning.gif]]

Figure 2. Peer tutoring in the Cognitive Tutor Algebra. Peer tutor's interface. 
[[Image:Walker_peer_tutoring.jpg]]

 Figure 3. Peer tutoring in the Cognitive Tutor Algebra. Adaptive support received by the peer tutor. 
[[Image:Walker_adaptive_support.jpg]]

=== Hypothesis ===
1. We expect the collaborative conditions to outperform the individual conditions in measures of robust learning of the algebra content.

2. We epect the peer tutoring plus cognitive tutoring condition to outperform the peer tutoring alone conditions in measures of robust learning of the algebra content

=== Dependent variables ===
* ''[[Normal post-test]]'': Students are given a post-test immediately after the study on isomorphic problems
* ''Far [[transfer]]'': This paper and pencil test assessed students' understanding of the main mathematical concepts from the learning phase. The transfer items students had to solve tapped the same knowledge components as the problems in instruction, however, the problems where non-isomorphic to those in the instruction, thus demanded students to flexibly apply their knowledge to problems with a new format.
* ''[[Accelerated future learning]] test'': Student learning on future equation solving units will be measured.

To compare [[collaboration skill]]s of students, we will be conducting an analysis of student dialogs during the learning phase.

To assess immediate effects of the instructional variations, we will analyze student progress on training problems as they work through the instruction.

=== Findings ===
'''Pre-study 1:'''

Our first study compared the peer tutoring to peer tutoring plus reflection conditions. Using pretest and posttest scores, we conducted a two-way (condition x test-time) repeated-measure ANOVA, with test-time as the repeated measure. Posttest scores were significantly higher than pretest scores in both the tutoring and the tutoring+reflection condition (F (1,12) = 15.25, p < .002, η² = 0.56), but there were no significant differences between conditions, and no interaction (see Table 2). To further examine what occurred during the collaboration phase we turned to log data and notes from classroom observation. During peer tutoring, students appeared engaged, and did exhibit many of the positive collaborative behaviors that we were attempting to encourage with our script and that have been shown to correlate with knowledge construction and self-reflection. However, we observed that peer tutors struggled to provide tutees with answers, and did not connect the preparation that they had done with the collaboration phase. For instance, they often did not consult their answer printouts when they did not know the next problem step and thus had to rely on teacher assistance to solve a problem. As a result, tutees skipped problems without completing them correctly. This undesirable behavior differed between the two conditions. Students in the tutoring condition attempted more problems than students in the tutoring+reflection condition, and appeared to complete more problems as well. The average number of problems completed by dyads in the tutoring+reflection condition was low; students in this group took an average of 11 minutes to complete a single problem, compared to a 6 minute average in the tutoring condition. Students in the tutoring condition tended to skip problems they could not solve, completing less than 60% of the problems they attempted. Immediately before skipping a problem, students would generally state their inability to solve it, “I don’t know how to do this one,“ or their lack of motivation, “Just do something and I’ll agree or something.” If students skip problems, they may not learn how to solve difficult problems. However, if they do not complete many problems, they may not be sufficiently exposed to all the skills involved in the unit, and will be given fewer opportunities to master them.

'''Study 1:'''
* All students learned as a result of the study. We conducted a two-way (condition x test-time) repeated-measure ANOVA, with test-time (pretest, posttest, or delayed test) as the repeated measure. There was a significant effect for test-time (F(2,72) = 41.303, p < .001), but there were no significant differences between conditions, and no interaction.
* Students in the collaborative conditions solved significantly fewer problems than students in the individual conditions. We conducted a one-way (condition: individual, fixed, adaptive) ANOVA on the number of problems completed per hour in the collaboration phase of the study (see Table 2). For this analysis, we grouped the students in the collaborative conditions by dyad, as the number of problems that one pair member completes (and the time that they take) is dependent on the number of problems the other pair member completes. Condition was indeed significantly related to problems solved (F(2,34) = 8.764, p = .001).
* There were some interesting lacks of difference in tutee and tutor behaviors within problems. For example, across all conditions, tutees tended to make the same number of mistakes per problem, request the same amount of help per problem, and receive the same amount of help from their tutors per problem
* There was an apparent relationship between tutee impasses and tutor learning. For example, the collaborative conditions differed on how easy it was for students to move to the next problem. In the adaptive condition, students could not continue unless they had successfully completed the problem, making it possible for students to get “stuck”, where they repeatedly tried incorrectly to move to the next problem. The number of these incorrect done tries was negatively correlated with tutee gain scores on the delayed test (r = -.591, p = .056), but positively correlated with tutor gain scores on the delayed test (r = .463, p = .115). In the fixed condition, students were not notified when their attempts to continue were incorrect, and thus could “skip” to the next problem even if the previous problem was not done. Problems skipped were negatively correlated with tutee learning (r = -.614, p = .059) and tutor learning (r = -.369, p = .329). If problems were skipped tutors did not benefit from tutee impasses.

=== Explanation ===
This study is part of the [[Interactive_Communication|Interactive Communication cluster]], and its hypothesis is a specialization of the IC cluster’s central hypothesis. According to the IC cluster’s hypothesis, instruction that yields [[robust learning]] should be designed to have the right target paths. Furthermore, it should improve the students’ path choice in the [[learning event space]], i.e., increasing the probability that students take correct paths and decreasing the probability that students take alternative paths.

By enhancing the Cognitive Tutor with collaboration, the study tried to reach these two goals in two steps:

* ''Individual condition'': The existing Cognitive Tutor already offers correct learning paths. However, individual learning on the Cognitive Tutor has several shortcomings. For example, students don’t have the opportunity to reflect on the underlying mathematical [[conceptual knowledge|concepts]] in natural language interaction in order to gain a deeper understanding. In addition, students do not always use the offered learning paths (as the opportunity to reflect on on-demand hints and error feedback) effectively, thus often do not engage in [[sense making]] processes
* ''Peer tutoring condition (1st step)'': By enhancing the Algebra I Cognitive Tutor to be a collaborative learning environment, we added a learning resource – the learning partner. This adds further correct learning paths to the [[learning event space]], for instance, learning by giving explanations, the possibility to request help, and learning by knowledge co-construction. However, similarly to the learning paths in an individual setting, students do not always capitalize on these learning opportunities. Further, if students lack the necessary expertise to tutor, they will not master the desired knowledge components, even if they take a correct learning path.
* ''Peer plus cognitive tutoring condition (2nd step)'': To increase the probability that students are mastering the correct knowledge components, the cognitive tutor helps peer tutors interact with their partners. Students use both the peer tutor and the cognitive tutor as learning resources.

=== Annotated bibliography ===
* Walker, E., Rummel, N., and Koedinger, K. R. To Tutor the Tutor: Adaptive Domain Support for Peer Tutoring. To appear at the 9th International Conference on Intelligent Tutoring Systems. 2008.
* Walker, E., McLaren, B. M., Rummel, N., and Koedinger, K. R. Who Says Three's a Crowd? Using a Cognitive Tutor to Support Peer Tutoring. To appear in the Proceedings of the 13th International Conference on Artificial Intelligence and Education. 2007.
* Walker, E., Rummel, N., McLaren, B. M. & Koedinger, K. R. The Student Becomes the Master: Integrating Peer Tutoring with Cognitive Tutoring. Short paper at the Conference on Computer Supported Collaborative Learning (CSCL-07). Rutgers University, July 16-21, 2007.
* Walker, E., Koedinger, K., McLaren, B. M., & Rummel, N. (2006). Cognitive tutors as research platforms: Extending an established tutoring system for collaborative and metacognitive experimentation. ''Lecture Notes in Computer Science, Volume 4053/2006. Proceedings of the 8th International Conference on Intelligent Tutoring Systems'' (pp. 207-216). Berlin: Springer
* Walker, E. (2005). Mutual peer tutoring: A collaborative addition to the Algebra-1 Cognitive Tutor. Paper presented at the 12th International Conference on Artificial Intelligence and Education (AIED-05, Young Researchers Track), July, 2005, Amsterdam, the Netherlands.

=== References ===

* Aleven, V., McLaren, B., Roll, I., & Koedinger, K. R. (2004). Toward tutoring help seeking: Applying cognitive modelling to meta-cognitive skills. In J. C. Lester, R. M. Vicari & F. Paraguaçu (Eds.), Proceedings of Seventh International Conference on Intelligent Tutoring Systems, ITS 2004 (pp. 227-239). Berlin: Springer.
* Anderson, J. R., Corbett, A. T., Koedinger, K. R., & Pelletier, R. (1995). Cognitive tutors: Lessons learned. Journal of the Learning Sciences, 4(2), 167-207.
* Baker, R. S., Corbett, A. T., & Koedinger, K. R. (2004). Detecting student misuse of intelligent tutoring systems. Paper presented at the Proceedings of the 7th International Conference on Intelligent Tutoring Systems.
* Berg, K. F. (1993). Structured cooperative learning and achievement in a high school mathematics class. Paper presented at the Annual Meeting of the American Educational Research Association, Atlanta, GA.
* Biswas, G., Schwartz, D. L., Leelawong, K., Vye, N., & TAG-V. (2005). Learning by teaching: A new agent paradigm for educational software. Applied Artificial Intelligence, 19, 363–392.
* Carmien, S., Kollar, I., Fischer, G. & Fischer, F. (2006). The interplay of internal and external scripts. In F. Fischer, I. Kollar, H. Mandl &, J. Haake, Scripting computer-supported communication of knowledge. Cognitive, computational, and educational perspectives (pp. 289-311). New York: Springer.
* Fantuzzo, J. W., Riggio, R. E., Connelly, S., & Dimeff, L. A. (1992). Effects of reciprocal peer tutoring on mathematics and school adjustment: A componential analysis. Journal of Educational Psychology, 84(3), 331-339.
* Fuchs, L.S., Fuchs, D., Prentice, K., Burch, M., Hamlett, C.L., Owen, R., & Schroeter, K. (2003). Enhancing third-grade students' mathematical problem solving with self-regulated learning strategies. Journal of Educational Psychology, 95(2), 306-315.
* Harrer, A., McLaren, B. M., Walker, E., Bollen, L., and Sewall, J. (2006). Creating Cognitive Tutors for Collaborative Learning: Steps Toward Realization. User Modeling and User-Adapted Interaction: The Journal of Personalization Research (UMUAI) (2006) 16: 175-209.
* Johnson, D. W. and Johnson, R. T. (1990). Cooperative learning and achievement. In S. Sharan (Ed.), Cooperative learning: Theory and research (pp. 23-37). New York: Praeger.
* King, A., Staffieri, A., & Adelgais, A. (1998). Mutual peer tutoring: Effects of structuring tutorial interaction to scaffold peer learning. Journal of Educational Psychology, 90, 134-152.
* Koedinger, K. R., Anderson, J. R., Hadley, W. H., & Mark, M. A. (1997). Intelligent tutoring goes to school in the big city. International Journal of Artificial Intelligence in Education, 8, 30-43.
* Kollar, I., Fischer, F., & Hesse, F. W. (in press). Collaboration scripts - a conceptual analysis. Educational Psychology Review.
* Ploetzner, R., Dillenbourg, P., Preier, M.,&Traum, D. (1999). Learning by explaining to oneself and to others. In P. Dillenbourg (Ed.), Collaborative learning. Cognitive and computational approaches (pp. 103–121). Amsterdam: Pergamon.
* Ritter, S., Blessing, S. B., & Hadley, W. S. (2002). SBIR Phase I Final Report 2002. Department of Education. Department of Education RFP ED: 84-305S.
* Rummel, N., & Spada, H. (2005). Learning to Collaborate: An Instructional Approach to Promoting Collaborative Problem Solving in Computer-Mediated Settings. Journal of the Learning Sciences, 14(2), 201-241.
* Soller, A.L. (2001). Supporting Social Interaction in an Intelligent Collaborative Learning System. International Journal of Artificial Intelligence in Education, 12, 40-62.
* Teasley, S. D. (1995). The role of talk in children's peer collaborations. Developmental Psychology, 31(2), 207-220.
* Webb, N.M., Troper, J.D., & Fall, R. (1995). Constructive activity and learning in collaborative small groups. Journal of Educational Psychology, 87, 406-423.

=== Connections ===
Like this study, [[Rummel Scripted Collaborative Problem Solving]] adds scripted collaborative problem solving to the Cognitive Tutor Algebra. The studies differ in the way collaboration is integrated in the Tutor. First, in the Rummel et al. study, both students first prepare one subtasks of a problem to mutually solve the complex story problem later on. Thus, although the students are experts for different parts of the problem, they have a comparable knowledge level during collaboration. In contrast, in this study, one student prepares to teach his partner. Then, they change roles. Thus, during collaboration, their knowledge level differs. Second, in the Rummel et al. study, collaboration was face to face, whereas this study used a chat tool for interaction.

Similar to the adaptive script component of the Collaborative Problem-Solving Script, the [[The Help Tutor Roll Aleven McLaren|Help Tutor project]] aims at improving students' [[help-seeking behavior]] and at reducing students' tendency to [[game the system]]. Furthermore, both studies contain instructions to teach [[metacognition]]. The metacognitive component in our study instructs students to monitor their interaction in order to improve it in subsequent collaborations; the Help Tutor project asks students to evaluate their need for help in order to improve their help-seeking behavior when learning on the Tutor.

Both in this study and in the [[Reflective Dialogues (Katz)|Reflective Dialogue study]] from Katz, students are asked to engage in reflection following each problem-solving. In this study, the reflection concentrates on the collaborative skills, while in Katz' study, the reflection concentrates on students' domain knowledge of the main principles applied in the problem.

Furthermore, both our study and the [[Help Lite (Aleven, Roll)]] aim at improving conceptual knowledge.

=== Future plans ===
Our future plans for June 2007 - December 2007:
* Run full study
* Analyze process data
* Conference contributions at CSCL and AIED
* Paper publication

[[Category:Study]]

Adaptive Assistance for Peer Tutoring (Walker, Rummer, Koedinger)

2008-12-09T01:41:40Z

Erin-Walker: /* Independent variables */

== Collaborative Extensions to the Cognitive Tutor Algebra: Adaptive Assistance for Peer Tutoring ==
''Erin Walker, Nikol Rummel, and Ken Koedinger''

=== Summary Tables ===
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''PI''' || Erin Walker
|-
| '''Co-PIs''' || Nikol Rummel, Ken Koedinger
|}
 

=== Abstract ===
Our research goal is to integrate a [[collaboration script|peer tutoring script]] within the context of the Cognitive Tutor Algebra (CTA) allowing students to tutor each other through the interface of an [[cognitive tutor|intelligent tutoring system (ITS)]] that provides both domain support and collaborative tutoring. In the PSLC project, “[[Walker A Peer Tutoring Addition|Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition]],” we have added peer tutoring to the Cognitive Tutor Algebra and developed adaptive domain support for the peer tutor. We propose to continue this work by developing and evaluating collaborative assistance for the peer tutor within this context. This assistance will target both the skills required to successfully tutor and the motivation for students to tutor. Once we have shown fixed collaborative assistance to be effective, we plan to implement it in an adaptive fashion, and compare the effects of adaptive and fixed assistance on collaborative skill acquisition and robust domain learning.

The integration of intelligent tutoring and collaborative learning allows us to investigate the differential effects of varying the type and adaptivity of assistance provided to collaborating peers on the acquisition of collaborative skills and on robust domain learning. Further, the development of a successful adaptive collaborative learning system would be a significant contribution to the ITS community.

=== Background and Significance ===

'''Peer Tutoring: Learning by Teaching'''

Incorporating peer tutoring into the CTA might be a way to encourage deep learning. Roscoe and Chi conclude that peer tutors benefit due to knowledge-building, where they reflect on their current knowledge and use it as a basis for constructing new knowledge (Roscoe & Chi, 2007). Because these positive effects are independent of tutor domain ability, researchers implement reciprocal peer tutoring programs, where students of similar abilities take turns tutoring each other. This type of peer tutoring has been shown to increase academic achievement and positive attitudes in long-term classroom interventions (Fantuzzo, Riggio, Connely, & Dimeff, 1989). Biswas et al. (2005) described three properties of peer tutoring related to tutor learning: tutors are accountable for their tutee’s knowledge, they reflect on tutee actions, and they engage in asking questions and giving explanations. Tutee learning is maximized at times when the tutee reaches an impasse, is prompted to find and explain the correct step, and is given an explanation if they fail to do so (VanLehn et al., 2003).

Peer tutors rarely exhibit knowledge-building behaviors spontaneously (Roscoe & Chi, 2007), and thus successful interventions provide them with assistance in order to achieve better learning outcomes for them and their tutees. This assistance can target tutoring behaviors through training, providing positive examples, or structuring the tutoring activities. For example, training students to give conceptual explanations had a significantly positive effect on learning (Fuchs et al., 1997). It is just as critical for assistance to target domain expertise of the peer tutors, in order to ensure that they have sufficient knowledge about a problem to help their partner solve it. Otherwise, there may be cognitive consequences (tutees cannot correctly solve problems) and affective consequences (students feel that they are poor tutors and become discouraged; Medway & Baron, 1997). Domain assistance can take the form of preparation on the problems and scaffolding during tutoring (e.g., Fantuzzo, Riggio, Connely, & Dimeff, 1989). Although assistance for peer tutoring has generally been fixed, providing adaptive support may be a promising approach.

'''Adaptive Collaborative Learning Systems'''

In order to benefit from collaboration students must interact in productive ways, and collaborative activities can be structured (scripted) to encourage these behaviors (e.g., Rummel & Spada, 2007). However, fixed scripts implemented in a one-size-fits-all fashion may be too restrictive for some students and place a high cognitive demand on others (Rummel & Spada, 2007; Dillenbourg, 2002). An adaptive system would be able to monitor student behaviors and provide support only when needed. Preliminary results suggest that adaptive support is indeed beneficial: Adaptive prompting realized in a Wizard of Oz fashion has been shown to have a positive effect on interaction and learning compared to an unscripted condition (Gweon, Rose, Carey, & Zaiss, 2006). An effective way to deliver this support would be to use an adaptive collaborative learning system, where feedback on collaboration is delivered by an intelligent agent.

Work on adaptive collaborative learning systems is still at an early stage. One approach is to use machine learning to detect problematic elements of student interaction in real-time and trigger helpful prompts. Although implementations have lead to significant learning gains, the adaptive feedback appears to be disruptive to dyadic interaction (Kumar et al., 2007). Another promising approach has explored using an intelligent agent as one of the collaborators; students teach the agent about ecosystems with the help of a mentoring agent (Biswas et al. 2005). However, the agents do not interact with the students in natural language, one of the primary benefits of collaboration.

With respect to peer tutoring, intelligent tutoring technology could be applied either to supporting tutor behaviors or domain knowledge of peer tutors. As it is very difficult to build an intelligent tutor for collaborative processes, we decided to develop a general script for the peer tutoring interaction and then focus on providing adaptive domain assistance to peer tutors by leveraging the existing domain models of the CTA. A condition where students tutor each other with adaptive domain support provided to the peer tutor is likely to be better than a condition where the peer tutor merely has access to an answer key, because the support would be tailored to each individual tutor’s needs. It is also likely to be better than a condition where students use the CTA individually, because the students in the collaborative condition would be able to interact deeply about the domain material.

=== Glossary ===
See [[:Category:Peer Tutoring|Peer Tutoring Glossary]]

=== Research Question ===
What are the differential effects of adaptive and fixed support on student collaborative process during a peer tutoring activity and the resulting [[robust learning]] outcomes?

How does an [[instructional method]] that provides metacognitive support and incentives for peer tutoring affect student collaborative process and [[robust learning]] outcomes?

=== Independent variables ===
We vary the [[agent]]s involved in the interaction using the [[instructional method|mode of instruction]]. For example, students may interact individually with a [[cognitive tutor]], may interact with each other using a [[collaboration script]], or may interact with each other with the help of a [[cognitive tutor]].

We also plan to examine the type of assistance provided to the students. They may receive assistance to tutoring competence (either domain, metacognitive, or procedural) or motivational support.

Currently, we vary whether students collaborate or work individually, and the adaptivity of the support they receive:

Figure 1. Individual learning in the Cognitive Tutor Algebra. 
[[Image:Walker_individual_learning.gif]]

Figure 2. Peer tutoring in the Cognitive Tutor Algebra. Peer tutor's interface. 
[[Image:Walker_peer_tutoring.jpg]]

 Figure 3. Peer tutoring in the Cognitive Tutor Algebra. Adaptive support received by the peer tutor. 
[[Image:Walker_adaptive_support.jpg]]

=== Hypotheses ===
1. Students that show high tutoring competence behaviors will show
more domain learning than students that show low tutoring competence behaviors

2. Student perceptions of their tutoring role and beliefs that they can fill
that role are related to their use of effective tutoring behaviors.

3. Either competence or motivational assistance alone is better than no
assistance at promoting good peer tutoring behaviors. Providing peer tutors with
both competence and motivational assistance will lead them to show better peer
tutoring behaviors than with only competence or motivational assistance.

4. An increase in positive peer tutoring behaviors due to fixed
assistance will lead to an increase in peer tutor robust domain learning

5. An increase in positive peer tutoring behaviors due to fixed assistance
will lead to an increase in peer tutee robust domain learning

6. Adaptive assistance is more effective than fixed assistance at
improving peer tutoring behaviors, which promotes robust domain learning of the
peer tutor and peer tutee.

=== Dependent variables ===
* ''[[Normal post-test]]'': Students are given a post-test immediately after the study on isomorphic problems
* ''Far [[transfer]]'': This paper and pencil test assessed students' understanding of the main mathematical concepts from the learning phase. The transfer items students had to solve tapped the same knowledge components as the problems in instruction, however, the problems where non-isomorphic to those in the instruction, thus demanded students to flexibly apply their knowledge to problems with a new format.
* ''[[Long-term retention]]'': Students will be given a test a month after the immediate posttest.
* ''[[Accelerated future learning]] test'': Student learning on future equation solving units will be measured.

To compare [[collaboration skill]]s of students, we will be conducting an analysis of student dialogs during the learning phase.

To assess immediate effects of the instructional variations, we will analyze student progress on training problems as they work through the instruction.

=== Findings ===
We are still in the design phase of this project.

=== Annotated bibliography ===
* Walker, E., Rummel, N., and Koedinger, K. R. To Tutor the Tutor: Adaptive Domain Support for Peer Tutoring. To appear at the 9th International Conference on Intelligent Tutoring Systems. 2008.
* Walker, E., McLaren, B. M., Rummel, N., and Koedinger, K. R. Who Says Three's a Crowd? Using a Cognitive Tutor to Support Peer Tutoring. 13th International Conference on Artificial Intelligence and Education. 2007.
* Walker, E., Rummel, N., McLaren, B. M. & Koedinger, K. R. The Student Becomes the Master: Integrating Peer Tutoring with Cognitive Tutoring. Short paper at the Conference on Computer Supported Collaborative Learning (CSCL-07). Rutgers University, July 16-21, 2007.
* Walker, E., Koedinger, K., McLaren, B. M., & Rummel, N. (2006). Cognitive tutors as research platforms: Extending an established tutoring system for collaborative and metacognitive experimentation. ''Lecture Notes in Computer Science, Volume 4053/2006. Proceedings of the 8th International Conference on Intelligent Tutoring Systems'' (pp. 207-216). Berlin: Springer
* Walker, E. (2005). Mutual peer tutoring: A collaborative addition to the Algebra-1 Cognitive Tutor. Paper presented at the 12th International Conference on Artificial Intelligence and Education (AIED-05, Young Researchers Track), July, 2005, Amsterdam, the Netherlands.

=== References ===
* Roscoe, R. D. & Chi, M. Understanding tutor learning: Knowledge-building and knowledge-telling in peer tutors’ explanations and questions. Review of Educational Research 77(4), 534-574 (2007)
* Fantuzzo, J. W., Riggio, R. E., Connelly, S., & Dimeff, L. A. Effects of reciprocal peer tutoring on academic achievement and psychological adjustment: A component analysis. Journal of Educational Psychology 81(2), 173-177 (1989)
* Biswas, G., Schwartz, D. L., Leelawong, K., Vye, N., & TAG-V. Learning by teaching: A new agent paradigm for educational software. Applied Artificial Intelligence 19, 363–392 (2005)
* VanLehn, K., Siler, S., Murray, C., Yamauchi, T., & Baggett, W. Why do only some events cause learning during human tutoring? Cognition and Instruction 21(3), 209-249 (2003)
* Fuchs, L., Fuchs, D., Hamlett, C., Phillips, N., Karns, K., & Dutka, S. Enhancing students’ helping behaviour during peer-mediated instruction with conceptual mathematical explanations. The Elementary School Journal 97(3), 223-249 (1997)
* Medway, F. & Baron, R. Locus of control and tutors’ instructional style. Contemporary Educational Psychology, 2, 298-310 (1997).
* Rummel, N. & Spada, H. Can people learn computer-mediated collaboration by following a script? In F. Fischer, I. Kollar, H. Mandl &, J. Haake, Scripting computer-supported communication of knowledge. Cognitive, computational, and educational perspectives (pp. 47-63). New York: Springer. (2007)
* Dillenbourg, P. Over-scripting CSCL: The risks of blending collaborative learning with instructional design. In P. A. Kirschner (Ed.), Three worlds of CSCL. Can we support CSCL (pp. 61-91). Heerlen: Open Universiteit Nederland. (2002)
* Gweon, G., Rosé, C., Carey, R. & Zaiss, Z. Providing Support for Adaptive Scripting in an On-Line Collaborative Learning Environment. Proc. of CHI 2006, pp. 251-260. (2006)
* Kumar, R., Rosé, C. P., Wang, Y. C., Joshi, M., Robinson, A. Tutorial dialogue as adaptive collaborative learning support. Proceedings of the 13th International Conference on Artificial Intelligence in Education (AIED 2007), Amsterdam: IOSPress. (2007)

=== Connections ===
This study is an extension of the PSLC project "[[Walker A Peer Tutoring Addition|Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition]]."

Like this study, [[Rummel Scripted Collaborative Problem Solving]] adds scripted collaborative problem solving to the Cognitive Tutor Algebra. The studies differ in the way collaboration is integrated in the Tutor. First, in the Rummel et al. study, both students first prepare one subtasks of a problem to mutually solve the complex story problem later on. Thus, although the students are experts for different parts of the problem, they have a comparable knowledge level during collaboration. In contrast, in this study, one student prepares to teach his partner. Then, they change roles. Thus, during collaboration, their knowledge level differs. Second, in the Rummel et al. study, collaboration was face to face, whereas this study used a chat tool for interaction.

Similar to the adaptive script component of the Collaborative Problem-Solving Script, the [[The Help Tutor Roll Aleven McLaren|Help Tutor project]] aims at improving students' [[help-seeking behavior]] and at reducing students' tendency to [[game the system]]. Furthermore, both studies contain instructions to teach [[metacognition]]. The metacognitive component in our study instructs students to monitor their interaction in order to improve it in subsequent collaborations; the Help Tutor project asks students to evaluate their need for help in order to improve their help-seeking behavior when learning on the Tutor.

Both in this study and in the [[Reflective Dialogues (Katz)|Reflective Dialogue study]] from Katz, students are asked to engage in reflection following each problem-solving. In this study, the reflection concentrates on the collaborative skills, while in Katz' study, the reflection concentrates on students' domain knowledge of the main principles applied in the problem.

Furthermore, both our study and the [[Help Lite (Aleven, Roll)]] aim at improving conceptual knowledge.

=== Future plans ===
Our future plans for June 2008 - December 2008:
* Design & implement further assistance for peer tutoring
* Run lab study evaluating the assistance

[[Category:Study]]

Adaptive Assistance for Peer Tutoring (Walker, Rummer, Koedinger)

2008-12-09T01:40:52Z

Erin-Walker: /* Independent variables */

== Collaborative Extensions to the Cognitive Tutor Algebra: Adaptive Assistance for Peer Tutoring ==
''Erin Walker, Nikol Rummel, and Ken Koedinger''

=== Summary Tables ===
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''PI''' || Erin Walker
|-
| '''Co-PIs''' || Nikol Rummel, Ken Koedinger
|}
 

=== Abstract ===
Our research goal is to integrate a [[collaboration script|peer tutoring script]] within the context of the Cognitive Tutor Algebra (CTA) allowing students to tutor each other through the interface of an [[cognitive tutor|intelligent tutoring system (ITS)]] that provides both domain support and collaborative tutoring. In the PSLC project, “[[Walker A Peer Tutoring Addition|Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition]],” we have added peer tutoring to the Cognitive Tutor Algebra and developed adaptive domain support for the peer tutor. We propose to continue this work by developing and evaluating collaborative assistance for the peer tutor within this context. This assistance will target both the skills required to successfully tutor and the motivation for students to tutor. Once we have shown fixed collaborative assistance to be effective, we plan to implement it in an adaptive fashion, and compare the effects of adaptive and fixed assistance on collaborative skill acquisition and robust domain learning.

The integration of intelligent tutoring and collaborative learning allows us to investigate the differential effects of varying the type and adaptivity of assistance provided to collaborating peers on the acquisition of collaborative skills and on robust domain learning. Further, the development of a successful adaptive collaborative learning system would be a significant contribution to the ITS community.

=== Background and Significance ===

'''Peer Tutoring: Learning by Teaching'''

Incorporating peer tutoring into the CTA might be a way to encourage deep learning. Roscoe and Chi conclude that peer tutors benefit due to knowledge-building, where they reflect on their current knowledge and use it as a basis for constructing new knowledge (Roscoe & Chi, 2007). Because these positive effects are independent of tutor domain ability, researchers implement reciprocal peer tutoring programs, where students of similar abilities take turns tutoring each other. This type of peer tutoring has been shown to increase academic achievement and positive attitudes in long-term classroom interventions (Fantuzzo, Riggio, Connely, & Dimeff, 1989). Biswas et al. (2005) described three properties of peer tutoring related to tutor learning: tutors are accountable for their tutee’s knowledge, they reflect on tutee actions, and they engage in asking questions and giving explanations. Tutee learning is maximized at times when the tutee reaches an impasse, is prompted to find and explain the correct step, and is given an explanation if they fail to do so (VanLehn et al., 2003).

Peer tutors rarely exhibit knowledge-building behaviors spontaneously (Roscoe & Chi, 2007), and thus successful interventions provide them with assistance in order to achieve better learning outcomes for them and their tutees. This assistance can target tutoring behaviors through training, providing positive examples, or structuring the tutoring activities. For example, training students to give conceptual explanations had a significantly positive effect on learning (Fuchs et al., 1997). It is just as critical for assistance to target domain expertise of the peer tutors, in order to ensure that they have sufficient knowledge about a problem to help their partner solve it. Otherwise, there may be cognitive consequences (tutees cannot correctly solve problems) and affective consequences (students feel that they are poor tutors and become discouraged; Medway & Baron, 1997). Domain assistance can take the form of preparation on the problems and scaffolding during tutoring (e.g., Fantuzzo, Riggio, Connely, & Dimeff, 1989). Although assistance for peer tutoring has generally been fixed, providing adaptive support may be a promising approach.

'''Adaptive Collaborative Learning Systems'''

In order to benefit from collaboration students must interact in productive ways, and collaborative activities can be structured (scripted) to encourage these behaviors (e.g., Rummel & Spada, 2007). However, fixed scripts implemented in a one-size-fits-all fashion may be too restrictive for some students and place a high cognitive demand on others (Rummel & Spada, 2007; Dillenbourg, 2002). An adaptive system would be able to monitor student behaviors and provide support only when needed. Preliminary results suggest that adaptive support is indeed beneficial: Adaptive prompting realized in a Wizard of Oz fashion has been shown to have a positive effect on interaction and learning compared to an unscripted condition (Gweon, Rose, Carey, & Zaiss, 2006). An effective way to deliver this support would be to use an adaptive collaborative learning system, where feedback on collaboration is delivered by an intelligent agent.

Work on adaptive collaborative learning systems is still at an early stage. One approach is to use machine learning to detect problematic elements of student interaction in real-time and trigger helpful prompts. Although implementations have lead to significant learning gains, the adaptive feedback appears to be disruptive to dyadic interaction (Kumar et al., 2007). Another promising approach has explored using an intelligent agent as one of the collaborators; students teach the agent about ecosystems with the help of a mentoring agent (Biswas et al. 2005). However, the agents do not interact with the students in natural language, one of the primary benefits of collaboration.

With respect to peer tutoring, intelligent tutoring technology could be applied either to supporting tutor behaviors or domain knowledge of peer tutors. As it is very difficult to build an intelligent tutor for collaborative processes, we decided to develop a general script for the peer tutoring interaction and then focus on providing adaptive domain assistance to peer tutors by leveraging the existing domain models of the CTA. A condition where students tutor each other with adaptive domain support provided to the peer tutor is likely to be better than a condition where the peer tutor merely has access to an answer key, because the support would be tailored to each individual tutor’s needs. It is also likely to be better than a condition where students use the CTA individually, because the students in the collaborative condition would be able to interact deeply about the domain material.

=== Glossary ===
See [[:Category:Peer Tutoring|Peer Tutoring Glossary]]

=== Research Question ===
What are the differential effects of adaptive and fixed support on student collaborative process during a peer tutoring activity and the resulting [[robust learning]] outcomes?

How does an [[instructional method]] that provides metacognitive support and incentives for peer tutoring affect student collaborative process and [[robust learning]] outcomes?

=== Independent variables ===
We vary the [[agent]]s involved in the interaction using the [[instructional method|mode of instruction]]. For example, students may interact individually with a [[cognitive tutor]], may interact with each other using a [[collaboration script]], or may interact with each other with the help of a [[cognitive tutor]].

We also plan to examine the type of assistance provided to the students. They may receive assistance to tutoring competence (either domain, metacognitive, or procedural) or motivational support.

Currently, we vary whether students collaborate or work individually, and the adaptivity of the support they receive:

Figure 1. Individual learning in the Cognitive Tutor Algebra. 
[[Image:Walker_individual_learning.gif]]

Figure 2. Peer tutoring in the Cognitive Tutor Algebra. Peer tutor's interface. 
[[Image:Walker_peer_tutoring.jpg]]

Figure 3. Peer tutoring in the Cognitive Tutor Algebra. Adaptive support received by the peer tutor. 
[[Image:Walker_adaptive_support.jpg]]

=== Hypotheses ===
1. Students that show high tutoring competence behaviors will show
more domain learning than students that show low tutoring competence behaviors

2. Student perceptions of their tutoring role and beliefs that they can fill
that role are related to their use of effective tutoring behaviors.

3. Either competence or motivational assistance alone is better than no
assistance at promoting good peer tutoring behaviors. Providing peer tutors with
both competence and motivational assistance will lead them to show better peer
tutoring behaviors than with only competence or motivational assistance.

4. An increase in positive peer tutoring behaviors due to fixed
assistance will lead to an increase in peer tutor robust domain learning

5. An increase in positive peer tutoring behaviors due to fixed assistance
will lead to an increase in peer tutee robust domain learning

6. Adaptive assistance is more effective than fixed assistance at
improving peer tutoring behaviors, which promotes robust domain learning of the
peer tutor and peer tutee.

=== Dependent variables ===
* ''[[Normal post-test]]'': Students are given a post-test immediately after the study on isomorphic problems
* ''Far [[transfer]]'': This paper and pencil test assessed students' understanding of the main mathematical concepts from the learning phase. The transfer items students had to solve tapped the same knowledge components as the problems in instruction, however, the problems where non-isomorphic to those in the instruction, thus demanded students to flexibly apply their knowledge to problems with a new format.
* ''[[Long-term retention]]'': Students will be given a test a month after the immediate posttest.
* ''[[Accelerated future learning]] test'': Student learning on future equation solving units will be measured.

To compare [[collaboration skill]]s of students, we will be conducting an analysis of student dialogs during the learning phase.

To assess immediate effects of the instructional variations, we will analyze student progress on training problems as they work through the instruction.

=== Findings ===
We are still in the design phase of this project.

=== Annotated bibliography ===
* Walker, E., Rummel, N., and Koedinger, K. R. To Tutor the Tutor: Adaptive Domain Support for Peer Tutoring. To appear at the 9th International Conference on Intelligent Tutoring Systems. 2008.
* Walker, E., McLaren, B. M., Rummel, N., and Koedinger, K. R. Who Says Three's a Crowd? Using a Cognitive Tutor to Support Peer Tutoring. 13th International Conference on Artificial Intelligence and Education. 2007.
* Walker, E., Rummel, N., McLaren, B. M. & Koedinger, K. R. The Student Becomes the Master: Integrating Peer Tutoring with Cognitive Tutoring. Short paper at the Conference on Computer Supported Collaborative Learning (CSCL-07). Rutgers University, July 16-21, 2007.
* Walker, E., Koedinger, K., McLaren, B. M., & Rummel, N. (2006). Cognitive tutors as research platforms: Extending an established tutoring system for collaborative and metacognitive experimentation. ''Lecture Notes in Computer Science, Volume 4053/2006. Proceedings of the 8th International Conference on Intelligent Tutoring Systems'' (pp. 207-216). Berlin: Springer
* Walker, E. (2005). Mutual peer tutoring: A collaborative addition to the Algebra-1 Cognitive Tutor. Paper presented at the 12th International Conference on Artificial Intelligence and Education (AIED-05, Young Researchers Track), July, 2005, Amsterdam, the Netherlands.

=== References ===
* Roscoe, R. D. & Chi, M. Understanding tutor learning: Knowledge-building and knowledge-telling in peer tutors’ explanations and questions. Review of Educational Research 77(4), 534-574 (2007)
* Fantuzzo, J. W., Riggio, R. E., Connelly, S., & Dimeff, L. A. Effects of reciprocal peer tutoring on academic achievement and psychological adjustment: A component analysis. Journal of Educational Psychology 81(2), 173-177 (1989)
* Biswas, G., Schwartz, D. L., Leelawong, K., Vye, N., & TAG-V. Learning by teaching: A new agent paradigm for educational software. Applied Artificial Intelligence 19, 363–392 (2005)
* VanLehn, K., Siler, S., Murray, C., Yamauchi, T., & Baggett, W. Why do only some events cause learning during human tutoring? Cognition and Instruction 21(3), 209-249 (2003)
* Fuchs, L., Fuchs, D., Hamlett, C., Phillips, N., Karns, K., & Dutka, S. Enhancing students’ helping behaviour during peer-mediated instruction with conceptual mathematical explanations. The Elementary School Journal 97(3), 223-249 (1997)
* Medway, F. & Baron, R. Locus of control and tutors’ instructional style. Contemporary Educational Psychology, 2, 298-310 (1997).
* Rummel, N. & Spada, H. Can people learn computer-mediated collaboration by following a script? In F. Fischer, I. Kollar, H. Mandl &, J. Haake, Scripting computer-supported communication of knowledge. Cognitive, computational, and educational perspectives (pp. 47-63). New York: Springer. (2007)
* Dillenbourg, P. Over-scripting CSCL: The risks of blending collaborative learning with instructional design. In P. A. Kirschner (Ed.), Three worlds of CSCL. Can we support CSCL (pp. 61-91). Heerlen: Open Universiteit Nederland. (2002)
* Gweon, G., Rosé, C., Carey, R. & Zaiss, Z. Providing Support for Adaptive Scripting in an On-Line Collaborative Learning Environment. Proc. of CHI 2006, pp. 251-260. (2006)
* Kumar, R., Rosé, C. P., Wang, Y. C., Joshi, M., Robinson, A. Tutorial dialogue as adaptive collaborative learning support. Proceedings of the 13th International Conference on Artificial Intelligence in Education (AIED 2007), Amsterdam: IOSPress. (2007)

=== Connections ===
This study is an extension of the PSLC project "[[Walker A Peer Tutoring Addition|Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition]]."

Like this study, [[Rummel Scripted Collaborative Problem Solving]] adds scripted collaborative problem solving to the Cognitive Tutor Algebra. The studies differ in the way collaboration is integrated in the Tutor. First, in the Rummel et al. study, both students first prepare one subtasks of a problem to mutually solve the complex story problem later on. Thus, although the students are experts for different parts of the problem, they have a comparable knowledge level during collaboration. In contrast, in this study, one student prepares to teach his partner. Then, they change roles. Thus, during collaboration, their knowledge level differs. Second, in the Rummel et al. study, collaboration was face to face, whereas this study used a chat tool for interaction.

Similar to the adaptive script component of the Collaborative Problem-Solving Script, the [[The Help Tutor Roll Aleven McLaren|Help Tutor project]] aims at improving students' [[help-seeking behavior]] and at reducing students' tendency to [[game the system]]. Furthermore, both studies contain instructions to teach [[metacognition]]. The metacognitive component in our study instructs students to monitor their interaction in order to improve it in subsequent collaborations; the Help Tutor project asks students to evaluate their need for help in order to improve their help-seeking behavior when learning on the Tutor.

Both in this study and in the [[Reflective Dialogues (Katz)|Reflective Dialogue study]] from Katz, students are asked to engage in reflection following each problem-solving. In this study, the reflection concentrates on the collaborative skills, while in Katz' study, the reflection concentrates on students' domain knowledge of the main principles applied in the problem.

Furthermore, both our study and the [[Help Lite (Aleven, Roll)]] aim at improving conceptual knowledge.

=== Future plans ===
Our future plans for June 2008 - December 2008:
* Design & implement further assistance for peer tutoring
* Run lab study evaluating the assistance

[[Category:Study]]

File:Walker adaptive support.jpg

2008-12-09T01:40:00Z

Erin-Walker: Adaptive domain support to the peer tutor.

Adaptive domain support to the peer tutor.

File:Walker peer tutoring.jpg

2008-12-09T01:39:02Z

Erin-Walker: Screenshot of peer tutor's interface.

Screenshot of peer tutor's interface.

Adaptive Assistance for Peer Tutoring (Walker, Rummer, Koedinger)

2008-12-09T01:36:01Z

Erin-Walker: /* Independent variables */

== Collaborative Extensions to the Cognitive Tutor Algebra: Adaptive Assistance for Peer Tutoring ==
''Erin Walker, Nikol Rummel, and Ken Koedinger''

=== Summary Tables ===
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''PI''' || Erin Walker
|-
| '''Co-PIs''' || Nikol Rummel, Ken Koedinger
|}
 

=== Abstract ===
Our research goal is to integrate a [[collaboration script|peer tutoring script]] within the context of the Cognitive Tutor Algebra (CTA) allowing students to tutor each other through the interface of an [[cognitive tutor|intelligent tutoring system (ITS)]] that provides both domain support and collaborative tutoring. In the PSLC project, “[[Walker A Peer Tutoring Addition|Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition]],” we have added peer tutoring to the Cognitive Tutor Algebra and developed adaptive domain support for the peer tutor. We propose to continue this work by developing and evaluating collaborative assistance for the peer tutor within this context. This assistance will target both the skills required to successfully tutor and the motivation for students to tutor. Once we have shown fixed collaborative assistance to be effective, we plan to implement it in an adaptive fashion, and compare the effects of adaptive and fixed assistance on collaborative skill acquisition and robust domain learning.

The integration of intelligent tutoring and collaborative learning allows us to investigate the differential effects of varying the type and adaptivity of assistance provided to collaborating peers on the acquisition of collaborative skills and on robust domain learning. Further, the development of a successful adaptive collaborative learning system would be a significant contribution to the ITS community.

=== Background and Significance ===

'''Peer Tutoring: Learning by Teaching'''

Incorporating peer tutoring into the CTA might be a way to encourage deep learning. Roscoe and Chi conclude that peer tutors benefit due to knowledge-building, where they reflect on their current knowledge and use it as a basis for constructing new knowledge (Roscoe & Chi, 2007). Because these positive effects are independent of tutor domain ability, researchers implement reciprocal peer tutoring programs, where students of similar abilities take turns tutoring each other. This type of peer tutoring has been shown to increase academic achievement and positive attitudes in long-term classroom interventions (Fantuzzo, Riggio, Connely, & Dimeff, 1989). Biswas et al. (2005) described three properties of peer tutoring related to tutor learning: tutors are accountable for their tutee’s knowledge, they reflect on tutee actions, and they engage in asking questions and giving explanations. Tutee learning is maximized at times when the tutee reaches an impasse, is prompted to find and explain the correct step, and is given an explanation if they fail to do so (VanLehn et al., 2003).

Peer tutors rarely exhibit knowledge-building behaviors spontaneously (Roscoe & Chi, 2007), and thus successful interventions provide them with assistance in order to achieve better learning outcomes for them and their tutees. This assistance can target tutoring behaviors through training, providing positive examples, or structuring the tutoring activities. For example, training students to give conceptual explanations had a significantly positive effect on learning (Fuchs et al., 1997). It is just as critical for assistance to target domain expertise of the peer tutors, in order to ensure that they have sufficient knowledge about a problem to help their partner solve it. Otherwise, there may be cognitive consequences (tutees cannot correctly solve problems) and affective consequences (students feel that they are poor tutors and become discouraged; Medway & Baron, 1997). Domain assistance can take the form of preparation on the problems and scaffolding during tutoring (e.g., Fantuzzo, Riggio, Connely, & Dimeff, 1989). Although assistance for peer tutoring has generally been fixed, providing adaptive support may be a promising approach.

'''Adaptive Collaborative Learning Systems'''

In order to benefit from collaboration students must interact in productive ways, and collaborative activities can be structured (scripted) to encourage these behaviors (e.g., Rummel & Spada, 2007). However, fixed scripts implemented in a one-size-fits-all fashion may be too restrictive for some students and place a high cognitive demand on others (Rummel & Spada, 2007; Dillenbourg, 2002). An adaptive system would be able to monitor student behaviors and provide support only when needed. Preliminary results suggest that adaptive support is indeed beneficial: Adaptive prompting realized in a Wizard of Oz fashion has been shown to have a positive effect on interaction and learning compared to an unscripted condition (Gweon, Rose, Carey, & Zaiss, 2006). An effective way to deliver this support would be to use an adaptive collaborative learning system, where feedback on collaboration is delivered by an intelligent agent.

Work on adaptive collaborative learning systems is still at an early stage. One approach is to use machine learning to detect problematic elements of student interaction in real-time and trigger helpful prompts. Although implementations have lead to significant learning gains, the adaptive feedback appears to be disruptive to dyadic interaction (Kumar et al., 2007). Another promising approach has explored using an intelligent agent as one of the collaborators; students teach the agent about ecosystems with the help of a mentoring agent (Biswas et al. 2005). However, the agents do not interact with the students in natural language, one of the primary benefits of collaboration.

With respect to peer tutoring, intelligent tutoring technology could be applied either to supporting tutor behaviors or domain knowledge of peer tutors. As it is very difficult to build an intelligent tutor for collaborative processes, we decided to develop a general script for the peer tutoring interaction and then focus on providing adaptive domain assistance to peer tutors by leveraging the existing domain models of the CTA. A condition where students tutor each other with adaptive domain support provided to the peer tutor is likely to be better than a condition where the peer tutor merely has access to an answer key, because the support would be tailored to each individual tutor’s needs. It is also likely to be better than a condition where students use the CTA individually, because the students in the collaborative condition would be able to interact deeply about the domain material.

=== Glossary ===
See [[:Category:Peer Tutoring|Peer Tutoring Glossary]]

=== Research Question ===
What are the differential effects of adaptive and fixed support on student collaborative process during a peer tutoring activity and the resulting [[robust learning]] outcomes?

How does an [[instructional method]] that provides metacognitive support and incentives for peer tutoring affect student collaborative process and [[robust learning]] outcomes?

=== Independent variables ===
We vary the [[agent]]s involved in the interaction using the [[instructional method|mode of instruction]]. For example, students may interact individually with a [[cognitive tutor]], may interact with each other using a [[collaboration script]], or may interact with each other with the help of a [[cognitive tutor]].

We also plan to examine the type of assistance provided to the students. They may receive assistance to tutoring competence (either domain, metacognitive, or procedural) or motivational support.

Currently, we vary whether students collaborate or work individually, and the adaptivity of the support they receive:

Figure 1. Individual learning in the Cognitive Tutor Algebra. 
[[Image:Walker_individual_learning.gif]]

Figure 2. Peer tutoring in the Cognitive Tutor Algebra. Peer tutor's interface. 
[[Image:Walker_peer_tutoring.gif]]

Figure 3. Peer tutoring in the Cognitive Tutor Algebra. Adaptive support received by the peer tutor. 
[[Image:Walker_adaptive_domain_support.gif]]

=== Hypotheses ===
1. Students that show high tutoring competence behaviors will show
more domain learning than students that show low tutoring competence behaviors

2. Student perceptions of their tutoring role and beliefs that they can fill
that role are related to their use of effective tutoring behaviors.

3. Either competence or motivational assistance alone is better than no
assistance at promoting good peer tutoring behaviors. Providing peer tutors with
both competence and motivational assistance will lead them to show better peer
tutoring behaviors than with only competence or motivational assistance.

4. An increase in positive peer tutoring behaviors due to fixed
assistance will lead to an increase in peer tutor robust domain learning

5. An increase in positive peer tutoring behaviors due to fixed assistance
will lead to an increase in peer tutee robust domain learning

6. Adaptive assistance is more effective than fixed assistance at
improving peer tutoring behaviors, which promotes robust domain learning of the
peer tutor and peer tutee.

=== Dependent variables ===
* ''[[Normal post-test]]'': Students are given a post-test immediately after the study on isomorphic problems
* ''Far [[transfer]]'': This paper and pencil test assessed students' understanding of the main mathematical concepts from the learning phase. The transfer items students had to solve tapped the same knowledge components as the problems in instruction, however, the problems where non-isomorphic to those in the instruction, thus demanded students to flexibly apply their knowledge to problems with a new format.
* ''[[Long-term retention]]'': Students will be given a test a month after the immediate posttest.
* ''[[Accelerated future learning]] test'': Student learning on future equation solving units will be measured.

To compare [[collaboration skill]]s of students, we will be conducting an analysis of student dialogs during the learning phase.

To assess immediate effects of the instructional variations, we will analyze student progress on training problems as they work through the instruction.

=== Findings ===
We are still in the design phase of this project.

=== Annotated bibliography ===
* Walker, E., Rummel, N., and Koedinger, K. R. To Tutor the Tutor: Adaptive Domain Support for Peer Tutoring. To appear at the 9th International Conference on Intelligent Tutoring Systems. 2008.
* Walker, E., McLaren, B. M., Rummel, N., and Koedinger, K. R. Who Says Three's a Crowd? Using a Cognitive Tutor to Support Peer Tutoring. 13th International Conference on Artificial Intelligence and Education. 2007.
* Walker, E., Rummel, N., McLaren, B. M. & Koedinger, K. R. The Student Becomes the Master: Integrating Peer Tutoring with Cognitive Tutoring. Short paper at the Conference on Computer Supported Collaborative Learning (CSCL-07). Rutgers University, July 16-21, 2007.
* Walker, E., Koedinger, K., McLaren, B. M., & Rummel, N. (2006). Cognitive tutors as research platforms: Extending an established tutoring system for collaborative and metacognitive experimentation. ''Lecture Notes in Computer Science, Volume 4053/2006. Proceedings of the 8th International Conference on Intelligent Tutoring Systems'' (pp. 207-216). Berlin: Springer
* Walker, E. (2005). Mutual peer tutoring: A collaborative addition to the Algebra-1 Cognitive Tutor. Paper presented at the 12th International Conference on Artificial Intelligence and Education (AIED-05, Young Researchers Track), July, 2005, Amsterdam, the Netherlands.

=== References ===
* Roscoe, R. D. & Chi, M. Understanding tutor learning: Knowledge-building and knowledge-telling in peer tutors’ explanations and questions. Review of Educational Research 77(4), 534-574 (2007)
* Fantuzzo, J. W., Riggio, R. E., Connelly, S., & Dimeff, L. A. Effects of reciprocal peer tutoring on academic achievement and psychological adjustment: A component analysis. Journal of Educational Psychology 81(2), 173-177 (1989)
* Biswas, G., Schwartz, D. L., Leelawong, K., Vye, N., & TAG-V. Learning by teaching: A new agent paradigm for educational software. Applied Artificial Intelligence 19, 363–392 (2005)
* VanLehn, K., Siler, S., Murray, C., Yamauchi, T., & Baggett, W. Why do only some events cause learning during human tutoring? Cognition and Instruction 21(3), 209-249 (2003)
* Fuchs, L., Fuchs, D., Hamlett, C., Phillips, N., Karns, K., & Dutka, S. Enhancing students’ helping behaviour during peer-mediated instruction with conceptual mathematical explanations. The Elementary School Journal 97(3), 223-249 (1997)
* Medway, F. & Baron, R. Locus of control and tutors’ instructional style. Contemporary Educational Psychology, 2, 298-310 (1997).
* Rummel, N. & Spada, H. Can people learn computer-mediated collaboration by following a script? In F. Fischer, I. Kollar, H. Mandl &, J. Haake, Scripting computer-supported communication of knowledge. Cognitive, computational, and educational perspectives (pp. 47-63). New York: Springer. (2007)
* Dillenbourg, P. Over-scripting CSCL: The risks of blending collaborative learning with instructional design. In P. A. Kirschner (Ed.), Three worlds of CSCL. Can we support CSCL (pp. 61-91). Heerlen: Open Universiteit Nederland. (2002)
* Gweon, G., Rosé, C., Carey, R. & Zaiss, Z. Providing Support for Adaptive Scripting in an On-Line Collaborative Learning Environment. Proc. of CHI 2006, pp. 251-260. (2006)
* Kumar, R., Rosé, C. P., Wang, Y. C., Joshi, M., Robinson, A. Tutorial dialogue as adaptive collaborative learning support. Proceedings of the 13th International Conference on Artificial Intelligence in Education (AIED 2007), Amsterdam: IOSPress. (2007)

=== Connections ===
This study is an extension of the PSLC project "[[Walker A Peer Tutoring Addition|Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition]]."

Like this study, [[Rummel Scripted Collaborative Problem Solving]] adds scripted collaborative problem solving to the Cognitive Tutor Algebra. The studies differ in the way collaboration is integrated in the Tutor. First, in the Rummel et al. study, both students first prepare one subtasks of a problem to mutually solve the complex story problem later on. Thus, although the students are experts for different parts of the problem, they have a comparable knowledge level during collaboration. In contrast, in this study, one student prepares to teach his partner. Then, they change roles. Thus, during collaboration, their knowledge level differs. Second, in the Rummel et al. study, collaboration was face to face, whereas this study used a chat tool for interaction.

Similar to the adaptive script component of the Collaborative Problem-Solving Script, the [[The Help Tutor Roll Aleven McLaren|Help Tutor project]] aims at improving students' [[help-seeking behavior]] and at reducing students' tendency to [[game the system]]. Furthermore, both studies contain instructions to teach [[metacognition]]. The metacognitive component in our study instructs students to monitor their interaction in order to improve it in subsequent collaborations; the Help Tutor project asks students to evaluate their need for help in order to improve their help-seeking behavior when learning on the Tutor.

Both in this study and in the [[Reflective Dialogues (Katz)|Reflective Dialogue study]] from Katz, students are asked to engage in reflection following each problem-solving. In this study, the reflection concentrates on the collaborative skills, while in Katz' study, the reflection concentrates on students' domain knowledge of the main principles applied in the problem.

Furthermore, both our study and the [[Help Lite (Aleven, Roll)]] aim at improving conceptual knowledge.

=== Future plans ===
Our future plans for June 2008 - December 2008:
* Design & implement further assistance for peer tutoring
* Run lab study evaluating the assistance

[[Category:Study]]

File:Walker individual learning.gif

2008-12-09T01:35:15Z

Erin-Walker: Screenshot of individual learning in literal equation solving using the Cognitive Tutor Algebra.

Screenshot of individual learning in literal equation solving using the Cognitive Tutor Algebra.

Walker A Peer Tutoring Addition

2008-04-14T19:57:34Z

Erin-Walker: /* Annotated bibliography */

== Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition ==
''Erin Walker, Bruce M. McLaren, Nikol Rummel, and Ken Koedinger''

=== Summary Tables ===
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''PIs''' || Bruce M. McLaren, Nikol Rummel, Ken Koedinger
|-
| '''Other Contributers''' ||
* Graduate Student: Erin Walker
* Staff: Jonathan Steinhart
|}
 
'' Pre Study 1''
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''Study Start Date''' || 05/06
|-
| '''Study End Date''' || 06/06
|-
| '''LearnLab Site''' || CWCTC
|-
| '''LearnLab Course''' || Algebra
|-
| '''Number of Students''' || ''N'' = 14
|-
| '''Average # of hours per participant''' || 3 hrs.
|}
 

'' Pre Study 2''
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''Study Start Date''' || 11/06
|-
| '''Study End Date''' || 11/06
|-
| '''LearnLab Site''' || CWCTC
|-
| '''LearnLab Course''' || Algebra
|-
| '''Number of Students''' || ''N'' = 20
|-
| '''Average # of hours per participant''' || 3 hrs.
|}
 

'' Full Study''
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''Study Start Date''' || 04/07
|-
| '''Study End Date''' || 04/07
|-
| '''LearnLab Site''' || CWCTC
|-
| '''LearnLab Course''' || Algebra
|-
| '''Number of Students''' || ''N'' = 70
|-
| '''Average # of hours per participant''' || 3 hrs.
|}
 

=== Abstract ===
In this project, the Algebra I Cognitive Tutor is extended to a [[collaborative learning environment]]: Instead of the computer tutoring a student, students tutor each other, and the [[cognitive tutor]] provides collaborative support. As research has shown, collaborative problem solving and learning has the potential to increase elaboration on the learning content. However, students are not always able to effectively meet the challenges of a collaborative setting. To ensure that students capitalize on collaborative problem solving with the Tutor, a [[peer tutoring]] [[collaboration scripts|collaboration script]] was developed that guides their interaction and prompts fruitful [[collaboration]].

In the baseline version of the script (without cognitive tutor support), students in the same class are seated at different computers and take turns tutoring each other. First, they prepare to tutor by solving problems using the regular cognitive tutor. Then, they tutor each other by marking each other right and wrong, monitoring their partner's skills, and give each other hints and feedback. After preparing and tutoring, they reflect on the problems and their interaction, engaging in [[metacognition|metacognitive activities]]. We have expanded the script to include intelligent cognitive support of the peer tutoring, and hope to ultimately expand it to include intelligent collaborative support.

To assess the effectiveness of the script on [[robust learning]], we are conducting a series of [[in vivo experiment]]s in the Algebra LearnLab. In an initial, small scale study (pre study) that served to establish basic effects and to test the procedure in a classroom setting, we compared peer tutoring with reflection elements to peer tutoring without reflection elements. In a second study that served to explore the effects of cognitive tutoring of peer tutoring, we compared peer tutoring plus cognitive tutoring to peer tutoring. In the full study, we will compare these collaborative conditions to an individual condition to assess the effect of the collaborative Tutor extension to regular Tutor use.

=== Background and Significance ===

In our project, we combined two different [[instructional method]]s both of which have been shown to improve students’ learning in mathematics: Learning with intelligent tutoring systems (Koedinger, Anderson, Hadley, & Mark, 1997) and collaborative problem solving (Berg, 1993). The [[Cognitive tutor|Cognitive Tutor]] Algebra that was used in our study is a tutor for mathematics instruction at the high school level. Students learn with the Tutor during part of the regular classroom sessions. The Tutor's main features are immediate error feedback, the possibility to ask for a hint when encountering impasses, and knowledge tracing, i.e. the Tutor creates and updates a model of the student’s knowledge and selects new problems tailored to the student’s knowledge level. Although several studies have proven its effectiveness, students do not always benefit from learning with the Tutor. First, because the Tutor places emphasis on learning [[procedural|procedural problem solving skills]], yet a deep understanding of underlying mathematical [[conceptual knowledge|concepts]] is not necessarily achieved (Anderson, Corbett, Koedinger, & Pelletier, 1995). Second, students do not always make good use of the learning opportunities provided by the Cognitive Tutor (e.g. [[help abuse]], see Aleven, McLaren, Roll, & Koedinger, 2004; [[game the system|gaming the system]], Baker, Corbett, & Koedinger, 2004). So far, the Cognitive Tutor has been used in an individual learning setting only. However, as research on collaborative learning has shown, collaboration can yield elaboration of learning content (Teasley, 1995), thus this could be a promising approach to reduce the Tutor’s shortcomings. On the other hand, students are not always able to effectively meet the challenges of a collaborative setting (Rummel & Spada, 2005). [[Collaboration scripts]] have proven effective in helping people meet the challenges encountered when learning or working collaboratively (Kollar, Fischer, & Hesse, in press).

Combining collaborative activities with intelligent tutoring might combine the benefits of both approaches. Collaborative interaction could augment the effects of an existing ITS by adding deeper interaction, and intelligent tutoring support could improve the quality of student collaboration by providing guidance to students as they attempt to follow a collaboration script. One way to incorporate collaboration into an ITS is to place the intelligent tutoring system in the role of one of the collaborators and have a student interact with this agent. For example, Biswas, Schwartz, Leelawong, Vye, and the TAG-V (2005) have the student collaborate with an agent (“Betty”) in a peer tutoring scenario. The student teaches Betty about ecosystems with the help of a second mentoring agent (“Mr. Davis”), and it has been shown that a student who takes the role of a teacher in this system benefits more than a student who takes the role of a learner in a parallel traditional tutoring system. However, this system does not yet support natural language interaction between the agents involved, and therefore does not take full advantage of important collaborative interaction mechanisms that can increase learning. Another way to combine collaboration and tutoring is to use the ITS to structure and tutor the student collaboration, by connecting collaborative tools with ITS components (see Harrer, McLaren, Walker, Bollen, & Sewall, 2006), and developing a model of good and ineffective collaboration in order to provide on-line hints and feedback (see Soller, 2001). Most research into computer tutoring of the collaboration between two students is still in a preliminary stage.

We integrate collaborative learning with an ITS using a peer tutoring framework. Similar to Biswas et al. (2005), we put the student in the role of the tutor, and intend to have the cognitive elements of the tutoring supported by another agent. However, unlike Biswas et al., (2005), the teachable agent is a real student. Therefore, we have intelligent tutoring components monitoring and providing feedback on student collaboration in addition to student cognition, like Harrer et al. (2006). Our ultimate goal is to allow students to tutor each other through the interface of an ITS, supported by both cognitive and collaborative tutoring.

=== Glossary ===
See [[:Category:Peer Tutoring|Peer Tutoring Glossary]]

=== Research question ===
Does [[collaboration]] – and in particular [[collaboration scripts|scripted collaboration]] – improve students’ [[robust learning]] in the domain of algebra?

Do [[instructional method|instructional activities]] that involve three agents (a student, a peer tutor, and a computer tutor) increase [[robust learning]] compared to instructional activities that involve two agents (student and peer tutor, student and computer tutor)?

=== Independent variables ===
We vary the [[agent]]s involved in the interaction using the [[instructional method|mode of instruction]]. There are four conditions:
* Individual condition: Students solved problems with the Algebra I Cognitive Tutor in the regular fashion (i.e. computer program as additional [[agent]]).
* Peer tutoring condition: Students tutored each other using the cognitive tutor interface (i.e., another peer as additional [[agent]]). They went through two phases:
** a preparation phase, where students prepared to tutor while solving problems with the Algebra I Cognitive Tutor in the regular fashion
** a collaboration phase, where students take turns tutoring each other. Peer tutors can mark answers right or wrong, monitor their partner's skills, and give hints and feedback
* Peer tutoring plus reflection: As in the peer tutoring condition, students go through a preparation phase and then tutor each other in the collaboration phase. However, students also go through a series of [[metacognition|metacognitive]] self-reflection activities to encourage them to engage in [[elaborative interaction]]
* Peer tutoring plus cognitive tutoring: Students participate in the peer tutoring plus reflection condition, but during the collaboration phase, peer tutors receive cognitive tutor hints and feedback (i.e., a peer and computer program as additional [[agent]]s). This feedback contains two components:
** a collaborative prompt that encourages students to interact with each other
** a cognitive hint targeted at the particular mistake the peer tutee is making

=== Hypothesis ===
1. We expect the collaborative conditions to outperform the individual conditions in measures of robust learning of the algebra content.

2. We epect the peer tutoring plus cognitive tutoring condition to outperform the peer tutoring alone conditions in measures of robust learning of the algebra content

=== Dependent variables ===
* ''[[Normal post-test]]'': Students are given a post-test immediately after the study on isomorphic problems
* ''Far [[transfer]]'': This paper and pencil test assessed students' understanding of the main mathematical concepts from the learning phase. The transfer items students had to solve tapped the same knowledge components as the problems in instruction, however, the problems where non-isomorphic to those in the instruction, thus demanded students to flexibly apply their knowledge to problems with a new format.
* ''[[Accelerated future learning]] test'': Student learning on future equation solving units will be measured.

To compare [[collaboration skill]]s of students, we will be conducting an analysis of student dialogs during the learning phase.

To assess immediate effects of the instructional variations, we will analyze student progress on training problems as they work through the instruction.

=== Findings ===
'''Pre-study 1:'''

Our first study compared the peer tutoring to peer tutoring plus reflection conditions. Using pretest and posttest scores, we conducted a two-way (condition x test-time) repeated-measure ANOVA, with test-time as the repeated measure. Posttest scores were significantly higher than pretest scores in both the tutoring and the tutoring+reflection condition (F (1,12) = 15.25, p < .002, η² = 0.56), but there were no significant differences between conditions, and no interaction (see Table 2). To further examine what occurred during the collaboration phase we turned to log data and notes from classroom observation. During peer tutoring, students appeared engaged, and did exhibit many of the positive collaborative behaviors that we were attempting to encourage with our script and that have been shown to correlate with knowledge construction and self-reflection. However, we observed that peer tutors struggled to provide tutees with answers, and did not connect the preparation that they had done with the collaboration phase. For instance, they often did not consult their answer printouts when they did not know the next problem step and thus had to rely on teacher assistance to solve a problem. As a result, tutees skipped problems without completing them correctly. This undesirable behavior differed between the two conditions. Students in the tutoring condition attempted more problems than students in the tutoring+reflection condition, and appeared to complete more problems as well. The average number of problems completed by dyads in the tutoring+reflection condition was low; students in this group took an average of 11 minutes to complete a single problem, compared to a 6 minute average in the tutoring condition. Students in the tutoring condition tended to skip problems they could not solve, completing less than 60% of the problems they attempted. Immediately before skipping a problem, students would generally state their inability to solve it, “I don’t know how to do this one,“ or their lack of motivation, “Just do something and I’ll agree or something.” If students skip problems, they may not learn how to solve difficult problems. However, if they do not complete many problems, they may not be sufficiently exposed to all the skills involved in the unit, and will be given fewer opportunities to master them.

'''Study 1:'''
* All students learned as a result of the study. We conducted a two-way (condition x test-time) repeated-measure ANOVA, with test-time (pretest, posttest, or delayed test) as the repeated measure. There was a significant effect for test-time (F(2,72) = 41.303, p < .001), but there were no significant differences between conditions, and no interaction.
* Students in the collaborative conditions solved significantly fewer problems than students in the individual conditions. We conducted a one-way (condition: individual, fixed, adaptive) ANOVA on the number of problems completed per hour in the collaboration phase of the study (see Table 2). For this analysis, we grouped the students in the collaborative conditions by dyad, as the number of problems that one pair member completes (and the time that they take) is dependent on the number of problems the other pair member completes. Condition was indeed significantly related to problems solved (F(2,34) = 8.764, p = .001).
* There were some interesting lacks of difference in tutee and tutor behaviors within problems. For example, across all conditions, tutees tended to make the same number of mistakes per problem, request the same amount of help per problem, and receive the same amount of help from their tutors per problem
* There was an apparent relationship between tutee impasses and tutor learning. For example, the collaborative conditions differed on how easy it was for students to move to the next problem. In the adaptive condition, students could not continue unless they had successfully completed the problem, making it possible for students to get “stuck”, where they repeatedly tried incorrectly to move to the next problem. The number of these incorrect done tries was negatively correlated with tutee gain scores on the delayed test (r = -.591, p = .056), but positively correlated with tutor gain scores on the delayed test (r = .463, p = .115). In the fixed condition, students were not notified when their attempts to continue were incorrect, and thus could “skip” to the next problem even if the previous problem was not done. Problems skipped were negatively correlated with tutee learning (r = -.614, p = .059) and tutor learning (r = -.369, p = .329). If problems were skipped tutors did not benefit from tutee impasses.

=== Explanation ===
This study is part of the [[Interactive_Communication|Interactive Communication cluster]], and its hypothesis is a specialization of the IC cluster’s central hypothesis. According to the IC cluster’s hypothesis, instruction that yields [[robust learning]] should be designed to have the right target paths. Furthermore, it should improve the students’ path choice in the [[learning event space]], i.e., increasing the probability that students take correct paths and decreasing the probability that students take alternative paths.

By enhancing the Cognitive Tutor with collaboration, the study tried to reach these two goals in two steps:

* ''Individual condition'': The existing Cognitive Tutor already offers correct learning paths. However, individual learning on the Cognitive Tutor has several shortcomings. For example, students don’t have the opportunity to reflect on the underlying mathematical [[conceptual knowledge|concepts]] in natural language interaction in order to gain a deeper understanding. In addition, students do not always use the offered learning paths (as the opportunity to reflect on on-demand hints and error feedback) effectively, thus often do not engage in [[sense making]] processes
* ''Peer tutoring condition (1st step)'': By enhancing the Algebra I Cognitive Tutor to be a collaborative learning environment, we added a learning resource – the learning partner. This adds further correct learning paths to the [[learning event space]], for instance, learning by giving explanations, the possibility to request help, and learning by knowledge co-construction. However, similarly to the learning paths in an individual setting, students do not always capitalize on these learning opportunities. Further, if students lack the necessary expertise to tutor, they will not master the desired knowledge components, even if they take a correct learning path.
* ''Peer plus cognitive tutoring condition (2nd step)'': To increase the probability that students are mastering the correct knowledge components, the cognitive tutor helps peer tutors interact with their partners. Students use both the peer tutor and the cognitive tutor as learning resources.

=== Annotated bibliography ===
* Walker, E., Rummel, N., and Koedinger, K. R. To Tutor the Tutor: Adaptive Domain Support for Peer Tutoring. To appear at the 9th International Conference on Intelligent Tutoring Systems. 2008.
* Walker, E., McLaren, B. M., Rummel, N., and Koedinger, K. R. Who Says Three's a Crowd? Using a Cognitive Tutor to Support Peer Tutoring. To appear in the Proceedings of the 13th International Conference on Artificial Intelligence and Education. 2007.
* Walker, E., Rummel, N., McLaren, B. M. & Koedinger, K. R. The Student Becomes the Master: Integrating Peer Tutoring with Cognitive Tutoring. Short paper at the Conference on Computer Supported Collaborative Learning (CSCL-07). Rutgers University, July 16-21, 2007.
* Walker, E., Koedinger, K., McLaren, B. M., & Rummel, N. (2006). Cognitive tutors as research platforms: Extending an established tutoring system for collaborative and metacognitive experimentation. ''Lecture Notes in Computer Science, Volume 4053/2006. Proceedings of the 8th International Conference on Intelligent Tutoring Systems'' (pp. 207-216). Berlin: Springer
* Walker, E. (2005). Mutual peer tutoring: A collaborative addition to the Algebra-1 Cognitive Tutor. Paper presented at the 12th International Conference on Artificial Intelligence and Education (AIED-05, Young Researchers Track), July, 2005, Amsterdam, the Netherlands.

=== References ===

* Aleven, V., McLaren, B., Roll, I., & Koedinger, K. R. (2004). Toward tutoring help seeking: Applying cognitive modelling to meta-cognitive skills. In J. C. Lester, R. M. Vicari & F. Paraguaçu (Eds.), Proceedings of Seventh International Conference on Intelligent Tutoring Systems, ITS 2004 (pp. 227-239). Berlin: Springer.
* Anderson, J. R., Corbett, A. T., Koedinger, K. R., & Pelletier, R. (1995). Cognitive tutors: Lessons learned. Journal of the Learning Sciences, 4(2), 167-207.
* Baker, R. S., Corbett, A. T., & Koedinger, K. R. (2004). Detecting student misuse of intelligent tutoring systems. Paper presented at the Proceedings of the 7th International Conference on Intelligent Tutoring Systems.
* Berg, K. F. (1993). Structured cooperative learning and achievement in a high school mathematics class. Paper presented at the Annual Meeting of the American Educational Research Association, Atlanta, GA.
* Biswas, G., Schwartz, D. L., Leelawong, K., Vye, N., & TAG-V. (2005). Learning by teaching: A new agent paradigm for educational software. Applied Artificial Intelligence, 19, 363–392.
* Carmien, S., Kollar, I., Fischer, G. & Fischer, F. (2006). The interplay of internal and external scripts. In F. Fischer, I. Kollar, H. Mandl &, J. Haake, Scripting computer-supported communication of knowledge. Cognitive, computational, and educational perspectives (pp. 289-311). New York: Springer.
* Fantuzzo, J. W., Riggio, R. E., Connelly, S., & Dimeff, L. A. (1992). Effects of reciprocal peer tutoring on mathematics and school adjustment: A componential analysis. Journal of Educational Psychology, 84(3), 331-339.
* Fuchs, L.S., Fuchs, D., Prentice, K., Burch, M., Hamlett, C.L., Owen, R., & Schroeter, K. (2003). Enhancing third-grade students' mathematical problem solving with self-regulated learning strategies. Journal of Educational Psychology, 95(2), 306-315.
* Harrer, A., McLaren, B. M., Walker, E., Bollen, L., and Sewall, J. (2006). Creating Cognitive Tutors for Collaborative Learning: Steps Toward Realization. User Modeling and User-Adapted Interaction: The Journal of Personalization Research (UMUAI) (2006) 16: 175-209.
* Johnson, D. W. and Johnson, R. T. (1990). Cooperative learning and achievement. In S. Sharan (Ed.), Cooperative learning: Theory and research (pp. 23-37). New York: Praeger.
* King, A., Staffieri, A., & Adelgais, A. (1998). Mutual peer tutoring: Effects of structuring tutorial interaction to scaffold peer learning. Journal of Educational Psychology, 90, 134-152.
* Koedinger, K. R., Anderson, J. R., Hadley, W. H., & Mark, M. A. (1997). Intelligent tutoring goes to school in the big city. International Journal of Artificial Intelligence in Education, 8, 30-43.
* Kollar, I., Fischer, F., & Hesse, F. W. (in press). Collaboration scripts - a conceptual analysis. Educational Psychology Review.
* Ploetzner, R., Dillenbourg, P., Preier, M.,&Traum, D. (1999). Learning by explaining to oneself and to others. In P. Dillenbourg (Ed.), Collaborative learning. Cognitive and computational approaches (pp. 103–121). Amsterdam: Pergamon.
* Ritter, S., Blessing, S. B., & Hadley, W. S. (2002). SBIR Phase I Final Report 2002. Department of Education. Department of Education RFP ED: 84-305S.
* Rummel, N., & Spada, H. (2005). Learning to Collaborate: An Instructional Approach to Promoting Collaborative Problem Solving in Computer-Mediated Settings. Journal of the Learning Sciences, 14(2), 201-241.
* Soller, A.L. (2001). Supporting Social Interaction in an Intelligent Collaborative Learning System. International Journal of Artificial Intelligence in Education, 12, 40-62.
* Teasley, S. D. (1995). The role of talk in children's peer collaborations. Developmental Psychology, 31(2), 207-220.
* Webb, N.M., Troper, J.D., & Fall, R. (1995). Constructive activity and learning in collaborative small groups. Journal of Educational Psychology, 87, 406-423.

=== Connections ===
Like this study, [[Rummel Scripted Collaborative Problem Solving]] adds scripted collaborative problem solving to the Cognitive Tutor Algebra. The studies differ in the way collaboration is integrated in the Tutor. First, in the Rummel et al. study, both students first prepare one subtasks of a problem to mutually solve the complex story problem later on. Thus, although the students are experts for different parts of the problem, they have a comparable knowledge level during collaboration. In contrast, in this study, one student prepares to teach his partner. Then, they change roles. Thus, during collaboration, their knowledge level differs. Second, in the Rummel et al. study, collaboration was face to face, whereas this study used a chat tool for interaction.

Similar to the adaptive script component of the Collaborative Problem-Solving Script, the [[The Help Tutor Roll Aleven McLaren|Help Tutor project]] aims at improving students' [[help-seeking behavior]] and at reducing students' tendency to [[game the system]]. Furthermore, both studies contain instructions to teach [[metacognition]]. The metacognitive component in our study instructs students to monitor their interaction in order to improve it in subsequent collaborations; the Help Tutor project asks students to evaluate their need for help in order to improve their help-seeking behavior when learning on the Tutor.

Both in this study and in the [[Reflective Dialogues (Katz)|Reflective Dialogue study]] from Katz, students are asked to engage in reflection following each problem-solving. In this study, the reflection concentrates on the collaborative skills, while in Katz' study, the reflection concentrates on students' domain knowledge of the main principles applied in the problem.

Furthermore, both our study and the [[Help Lite (Aleven, Roll)]] aim at improving conceptual knowledge.

=== Future plans ===
Our future plans for June 2007 - December 2007:
* Run full study
* Analyze process data
* Conference contributions at CSCL and AIED
* Paper publication

[[Category:Study]]

Walker A Peer Tutoring Addition

2008-04-14T19:56:38Z

Erin-Walker:

== Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition ==
''Erin Walker, Bruce M. McLaren, Nikol Rummel, and Ken Koedinger''

=== Summary Tables ===
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''PIs''' || Bruce M. McLaren, Nikol Rummel, Ken Koedinger
|-
| '''Other Contributers''' ||
* Graduate Student: Erin Walker
* Staff: Jonathan Steinhart
|}
 
'' Pre Study 1''
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''Study Start Date''' || 05/06
|-
| '''Study End Date''' || 06/06
|-
| '''LearnLab Site''' || CWCTC
|-
| '''LearnLab Course''' || Algebra
|-
| '''Number of Students''' || ''N'' = 14
|-
| '''Average # of hours per participant''' || 3 hrs.
|}
 

'' Pre Study 2''
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''Study Start Date''' || 11/06
|-
| '''Study End Date''' || 11/06
|-
| '''LearnLab Site''' || CWCTC
|-
| '''LearnLab Course''' || Algebra
|-
| '''Number of Students''' || ''N'' = 20
|-
| '''Average # of hours per participant''' || 3 hrs.
|}
 

'' Full Study''
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''Study Start Date''' || 04/07
|-
| '''Study End Date''' || 04/07
|-
| '''LearnLab Site''' || CWCTC
|-
| '''LearnLab Course''' || Algebra
|-
| '''Number of Students''' || ''N'' = 70
|-
| '''Average # of hours per participant''' || 3 hrs.
|}
 

=== Abstract ===
In this project, the Algebra I Cognitive Tutor is extended to a [[collaborative learning environment]]: Instead of the computer tutoring a student, students tutor each other, and the [[cognitive tutor]] provides collaborative support. As research has shown, collaborative problem solving and learning has the potential to increase elaboration on the learning content. However, students are not always able to effectively meet the challenges of a collaborative setting. To ensure that students capitalize on collaborative problem solving with the Tutor, a [[peer tutoring]] [[collaboration scripts|collaboration script]] was developed that guides their interaction and prompts fruitful [[collaboration]].

In the baseline version of the script (without cognitive tutor support), students in the same class are seated at different computers and take turns tutoring each other. First, they prepare to tutor by solving problems using the regular cognitive tutor. Then, they tutor each other by marking each other right and wrong, monitoring their partner's skills, and give each other hints and feedback. After preparing and tutoring, they reflect on the problems and their interaction, engaging in [[metacognition|metacognitive activities]]. We have expanded the script to include intelligent cognitive support of the peer tutoring, and hope to ultimately expand it to include intelligent collaborative support.

To assess the effectiveness of the script on [[robust learning]], we are conducting a series of [[in vivo experiment]]s in the Algebra LearnLab. In an initial, small scale study (pre study) that served to establish basic effects and to test the procedure in a classroom setting, we compared peer tutoring with reflection elements to peer tutoring without reflection elements. In a second study that served to explore the effects of cognitive tutoring of peer tutoring, we compared peer tutoring plus cognitive tutoring to peer tutoring. In the full study, we will compare these collaborative conditions to an individual condition to assess the effect of the collaborative Tutor extension to regular Tutor use.

=== Background and Significance ===

In our project, we combined two different [[instructional method]]s both of which have been shown to improve students’ learning in mathematics: Learning with intelligent tutoring systems (Koedinger, Anderson, Hadley, & Mark, 1997) and collaborative problem solving (Berg, 1993). The [[Cognitive tutor|Cognitive Tutor]] Algebra that was used in our study is a tutor for mathematics instruction at the high school level. Students learn with the Tutor during part of the regular classroom sessions. The Tutor's main features are immediate error feedback, the possibility to ask for a hint when encountering impasses, and knowledge tracing, i.e. the Tutor creates and updates a model of the student’s knowledge and selects new problems tailored to the student’s knowledge level. Although several studies have proven its effectiveness, students do not always benefit from learning with the Tutor. First, because the Tutor places emphasis on learning [[procedural|procedural problem solving skills]], yet a deep understanding of underlying mathematical [[conceptual knowledge|concepts]] is not necessarily achieved (Anderson, Corbett, Koedinger, & Pelletier, 1995). Second, students do not always make good use of the learning opportunities provided by the Cognitive Tutor (e.g. [[help abuse]], see Aleven, McLaren, Roll, & Koedinger, 2004; [[game the system|gaming the system]], Baker, Corbett, & Koedinger, 2004). So far, the Cognitive Tutor has been used in an individual learning setting only. However, as research on collaborative learning has shown, collaboration can yield elaboration of learning content (Teasley, 1995), thus this could be a promising approach to reduce the Tutor’s shortcomings. On the other hand, students are not always able to effectively meet the challenges of a collaborative setting (Rummel & Spada, 2005). [[Collaboration scripts]] have proven effective in helping people meet the challenges encountered when learning or working collaboratively (Kollar, Fischer, & Hesse, in press).

Combining collaborative activities with intelligent tutoring might combine the benefits of both approaches. Collaborative interaction could augment the effects of an existing ITS by adding deeper interaction, and intelligent tutoring support could improve the quality of student collaboration by providing guidance to students as they attempt to follow a collaboration script. One way to incorporate collaboration into an ITS is to place the intelligent tutoring system in the role of one of the collaborators and have a student interact with this agent. For example, Biswas, Schwartz, Leelawong, Vye, and the TAG-V (2005) have the student collaborate with an agent (“Betty”) in a peer tutoring scenario. The student teaches Betty about ecosystems with the help of a second mentoring agent (“Mr. Davis”), and it has been shown that a student who takes the role of a teacher in this system benefits more than a student who takes the role of a learner in a parallel traditional tutoring system. However, this system does not yet support natural language interaction between the agents involved, and therefore does not take full advantage of important collaborative interaction mechanisms that can increase learning. Another way to combine collaboration and tutoring is to use the ITS to structure and tutor the student collaboration, by connecting collaborative tools with ITS components (see Harrer, McLaren, Walker, Bollen, & Sewall, 2006), and developing a model of good and ineffective collaboration in order to provide on-line hints and feedback (see Soller, 2001). Most research into computer tutoring of the collaboration between two students is still in a preliminary stage.

We integrate collaborative learning with an ITS using a peer tutoring framework. Similar to Biswas et al. (2005), we put the student in the role of the tutor, and intend to have the cognitive elements of the tutoring supported by another agent. However, unlike Biswas et al., (2005), the teachable agent is a real student. Therefore, we have intelligent tutoring components monitoring and providing feedback on student collaboration in addition to student cognition, like Harrer et al. (2006). Our ultimate goal is to allow students to tutor each other through the interface of an ITS, supported by both cognitive and collaborative tutoring.

=== Glossary ===
See [[:Category:Peer Tutoring|Peer Tutoring Glossary]]

=== Research question ===
Does [[collaboration]] – and in particular [[collaboration scripts|scripted collaboration]] – improve students’ [[robust learning]] in the domain of algebra?

Do [[instructional method|instructional activities]] that involve three agents (a student, a peer tutor, and a computer tutor) increase [[robust learning]] compared to instructional activities that involve two agents (student and peer tutor, student and computer tutor)?

=== Independent variables ===
We vary the [[agent]]s involved in the interaction using the [[instructional method|mode of instruction]]. There are four conditions:
* Individual condition: Students solved problems with the Algebra I Cognitive Tutor in the regular fashion (i.e. computer program as additional [[agent]]).
* Peer tutoring condition: Students tutored each other using the cognitive tutor interface (i.e., another peer as additional [[agent]]). They went through two phases:
** a preparation phase, where students prepared to tutor while solving problems with the Algebra I Cognitive Tutor in the regular fashion
** a collaboration phase, where students take turns tutoring each other. Peer tutors can mark answers right or wrong, monitor their partner's skills, and give hints and feedback
* Peer tutoring plus reflection: As in the peer tutoring condition, students go through a preparation phase and then tutor each other in the collaboration phase. However, students also go through a series of [[metacognition|metacognitive]] self-reflection activities to encourage them to engage in [[elaborative interaction]]
* Peer tutoring plus cognitive tutoring: Students participate in the peer tutoring plus reflection condition, but during the collaboration phase, peer tutors receive cognitive tutor hints and feedback (i.e., a peer and computer program as additional [[agent]]s). This feedback contains two components:
** a collaborative prompt that encourages students to interact with each other
** a cognitive hint targeted at the particular mistake the peer tutee is making

=== Hypothesis ===
1. We expect the collaborative conditions to outperform the individual conditions in measures of robust learning of the algebra content.

2. We epect the peer tutoring plus cognitive tutoring condition to outperform the peer tutoring alone conditions in measures of robust learning of the algebra content

=== Dependent variables ===
* ''[[Normal post-test]]'': Students are given a post-test immediately after the study on isomorphic problems
* ''Far [[transfer]]'': This paper and pencil test assessed students' understanding of the main mathematical concepts from the learning phase. The transfer items students had to solve tapped the same knowledge components as the problems in instruction, however, the problems where non-isomorphic to those in the instruction, thus demanded students to flexibly apply their knowledge to problems with a new format.
* ''[[Accelerated future learning]] test'': Student learning on future equation solving units will be measured.

To compare [[collaboration skill]]s of students, we will be conducting an analysis of student dialogs during the learning phase.

To assess immediate effects of the instructional variations, we will analyze student progress on training problems as they work through the instruction.

=== Findings ===
'''Pre-study 1:'''

Our first study compared the peer tutoring to peer tutoring plus reflection conditions. Using pretest and posttest scores, we conducted a two-way (condition x test-time) repeated-measure ANOVA, with test-time as the repeated measure. Posttest scores were significantly higher than pretest scores in both the tutoring and the tutoring+reflection condition (F (1,12) = 15.25, p < .002, η² = 0.56), but there were no significant differences between conditions, and no interaction (see Table 2). To further examine what occurred during the collaboration phase we turned to log data and notes from classroom observation. During peer tutoring, students appeared engaged, and did exhibit many of the positive collaborative behaviors that we were attempting to encourage with our script and that have been shown to correlate with knowledge construction and self-reflection. However, we observed that peer tutors struggled to provide tutees with answers, and did not connect the preparation that they had done with the collaboration phase. For instance, they often did not consult their answer printouts when they did not know the next problem step and thus had to rely on teacher assistance to solve a problem. As a result, tutees skipped problems without completing them correctly. This undesirable behavior differed between the two conditions. Students in the tutoring condition attempted more problems than students in the tutoring+reflection condition, and appeared to complete more problems as well. The average number of problems completed by dyads in the tutoring+reflection condition was low; students in this group took an average of 11 minutes to complete a single problem, compared to a 6 minute average in the tutoring condition. Students in the tutoring condition tended to skip problems they could not solve, completing less than 60% of the problems they attempted. Immediately before skipping a problem, students would generally state their inability to solve it, “I don’t know how to do this one,“ or their lack of motivation, “Just do something and I’ll agree or something.” If students skip problems, they may not learn how to solve difficult problems. However, if they do not complete many problems, they may not be sufficiently exposed to all the skills involved in the unit, and will be given fewer opportunities to master them.

'''Study 1:'''
* All students learned as a result of the study. We conducted a two-way (condition x test-time) repeated-measure ANOVA, with test-time (pretest, posttest, or delayed test) as the repeated measure. There was a significant effect for test-time (F(2,72) = 41.303, p < .001), but there were no significant differences between conditions, and no interaction.
* Students in the collaborative conditions solved significantly fewer problems than students in the individual conditions. We conducted a one-way (condition: individual, fixed, adaptive) ANOVA on the number of problems completed per hour in the collaboration phase of the study (see Table 2). For this analysis, we grouped the students in the collaborative conditions by dyad, as the number of problems that one pair member completes (and the time that they take) is dependent on the number of problems the other pair member completes. Condition was indeed significantly related to problems solved (F(2,34) = 8.764, p = .001).
* There were some interesting lacks of difference in tutee and tutor behaviors within problems. For example, across all conditions, tutees tended to make the same number of mistakes per problem, request the same amount of help per problem, and receive the same amount of help from their tutors per problem
* There was an apparent relationship between tutee impasses and tutor learning. For example, the collaborative conditions differed on how easy it was for students to move to the next problem. In the adaptive condition, students could not continue unless they had successfully completed the problem, making it possible for students to get “stuck”, where they repeatedly tried incorrectly to move to the next problem. The number of these incorrect done tries was negatively correlated with tutee gain scores on the delayed test (r = -.591, p = .056), but positively correlated with tutor gain scores on the delayed test (r = .463, p = .115). In the fixed condition, students were not notified when their attempts to continue were incorrect, and thus could “skip” to the next problem even if the previous problem was not done. Problems skipped were negatively correlated with tutee learning (r = -.614, p = .059) and tutor learning (r = -.369, p = .329). If problems were skipped tutors did not benefit from tutee impasses.

=== Explanation ===
This study is part of the [[Interactive_Communication|Interactive Communication cluster]], and its hypothesis is a specialization of the IC cluster’s central hypothesis. According to the IC cluster’s hypothesis, instruction that yields [[robust learning]] should be designed to have the right target paths. Furthermore, it should improve the students’ path choice in the [[learning event space]], i.e., increasing the probability that students take correct paths and decreasing the probability that students take alternative paths.

By enhancing the Cognitive Tutor with collaboration, the study tried to reach these two goals in two steps:

* ''Individual condition'': The existing Cognitive Tutor already offers correct learning paths. However, individual learning on the Cognitive Tutor has several shortcomings. For example, students don’t have the opportunity to reflect on the underlying mathematical [[conceptual knowledge|concepts]] in natural language interaction in order to gain a deeper understanding. In addition, students do not always use the offered learning paths (as the opportunity to reflect on on-demand hints and error feedback) effectively, thus often do not engage in [[sense making]] processes
* ''Peer tutoring condition (1st step)'': By enhancing the Algebra I Cognitive Tutor to be a collaborative learning environment, we added a learning resource – the learning partner. This adds further correct learning paths to the [[learning event space]], for instance, learning by giving explanations, the possibility to request help, and learning by knowledge co-construction. However, similarly to the learning paths in an individual setting, students do not always capitalize on these learning opportunities. Further, if students lack the necessary expertise to tutor, they will not master the desired knowledge components, even if they take a correct learning path.
* ''Peer plus cognitive tutoring condition (2nd step)'': To increase the probability that students are mastering the correct knowledge components, the cognitive tutor helps peer tutors interact with their partners. Students use both the peer tutor and the cognitive tutor as learning resources.

=== Annotated bibliography ===
* Walker, E., McLaren, B. M., Rummel, N., and Koedinger, K. R. Who Says Three's a Crowd? Using a Cognitive Tutor to Support Peer Tutoring. To appear in the Proceedings of the 13th International Conference on Artificial Intelligence and Education. 2007.
* Walker, E., Rummel, N., McLaren, B. M. & Koedinger, K. R. The Student Becomes the Master: Integrating Peer Tutoring with Cognitive Tutoring. Short paper at the Conference on Computer Supported Collaborative Learning (CSCL-07). Rutgers University, July 16-21, 2007.
* Walker, E., Koedinger, K., McLaren, B. M., & Rummel, N. (2006). Cognitive tutors as research platforms: Extending an established tutoring system for collaborative and metacognitive experimentation. ''Lecture Notes in Computer Science, Volume 4053/2006. Proceedings of the 8th International Conference on Intelligent Tutoring Systems'' (pp. 207-216). Berlin: Springer
* Walker, E. (2005). Mutual peer tutoring: A collaborative addition to the Algebra-1 Cognitive Tutor. Paper presented at the 12th International Conference on Artificial Intelligence and Education (AIED-05, Young Researchers Track), July, 2005, Amsterdam, the Netherlands.

=== References ===

* Aleven, V., McLaren, B., Roll, I., & Koedinger, K. R. (2004). Toward tutoring help seeking: Applying cognitive modelling to meta-cognitive skills. In J. C. Lester, R. M. Vicari & F. Paraguaçu (Eds.), Proceedings of Seventh International Conference on Intelligent Tutoring Systems, ITS 2004 (pp. 227-239). Berlin: Springer.
* Anderson, J. R., Corbett, A. T., Koedinger, K. R., & Pelletier, R. (1995). Cognitive tutors: Lessons learned. Journal of the Learning Sciences, 4(2), 167-207.
* Baker, R. S., Corbett, A. T., & Koedinger, K. R. (2004). Detecting student misuse of intelligent tutoring systems. Paper presented at the Proceedings of the 7th International Conference on Intelligent Tutoring Systems.
* Berg, K. F. (1993). Structured cooperative learning and achievement in a high school mathematics class. Paper presented at the Annual Meeting of the American Educational Research Association, Atlanta, GA.
* Biswas, G., Schwartz, D. L., Leelawong, K., Vye, N., & TAG-V. (2005). Learning by teaching: A new agent paradigm for educational software. Applied Artificial Intelligence, 19, 363–392.
* Carmien, S., Kollar, I., Fischer, G. & Fischer, F. (2006). The interplay of internal and external scripts. In F. Fischer, I. Kollar, H. Mandl &, J. Haake, Scripting computer-supported communication of knowledge. Cognitive, computational, and educational perspectives (pp. 289-311). New York: Springer.
* Fantuzzo, J. W., Riggio, R. E., Connelly, S., & Dimeff, L. A. (1992). Effects of reciprocal peer tutoring on mathematics and school adjustment: A componential analysis. Journal of Educational Psychology, 84(3), 331-339.
* Fuchs, L.S., Fuchs, D., Prentice, K., Burch, M., Hamlett, C.L., Owen, R., & Schroeter, K. (2003). Enhancing third-grade students' mathematical problem solving with self-regulated learning strategies. Journal of Educational Psychology, 95(2), 306-315.
* Harrer, A., McLaren, B. M., Walker, E., Bollen, L., and Sewall, J. (2006). Creating Cognitive Tutors for Collaborative Learning: Steps Toward Realization. User Modeling and User-Adapted Interaction: The Journal of Personalization Research (UMUAI) (2006) 16: 175-209.
* Johnson, D. W. and Johnson, R. T. (1990). Cooperative learning and achievement. In S. Sharan (Ed.), Cooperative learning: Theory and research (pp. 23-37). New York: Praeger.
* King, A., Staffieri, A., & Adelgais, A. (1998). Mutual peer tutoring: Effects of structuring tutorial interaction to scaffold peer learning. Journal of Educational Psychology, 90, 134-152.
* Koedinger, K. R., Anderson, J. R., Hadley, W. H., & Mark, M. A. (1997). Intelligent tutoring goes to school in the big city. International Journal of Artificial Intelligence in Education, 8, 30-43.
* Kollar, I., Fischer, F., & Hesse, F. W. (in press). Collaboration scripts - a conceptual analysis. Educational Psychology Review.
* Ploetzner, R., Dillenbourg, P., Preier, M.,&Traum, D. (1999). Learning by explaining to oneself and to others. In P. Dillenbourg (Ed.), Collaborative learning. Cognitive and computational approaches (pp. 103–121). Amsterdam: Pergamon.
* Ritter, S., Blessing, S. B., & Hadley, W. S. (2002). SBIR Phase I Final Report 2002. Department of Education. Department of Education RFP ED: 84-305S.
* Rummel, N., & Spada, H. (2005). Learning to Collaborate: An Instructional Approach to Promoting Collaborative Problem Solving in Computer-Mediated Settings. Journal of the Learning Sciences, 14(2), 201-241.
* Soller, A.L. (2001). Supporting Social Interaction in an Intelligent Collaborative Learning System. International Journal of Artificial Intelligence in Education, 12, 40-62.
* Teasley, S. D. (1995). The role of talk in children's peer collaborations. Developmental Psychology, 31(2), 207-220.
* Webb, N.M., Troper, J.D., & Fall, R. (1995). Constructive activity and learning in collaborative small groups. Journal of Educational Psychology, 87, 406-423.

=== Connections ===
Like this study, [[Rummel Scripted Collaborative Problem Solving]] adds scripted collaborative problem solving to the Cognitive Tutor Algebra. The studies differ in the way collaboration is integrated in the Tutor. First, in the Rummel et al. study, both students first prepare one subtasks of a problem to mutually solve the complex story problem later on. Thus, although the students are experts for different parts of the problem, they have a comparable knowledge level during collaboration. In contrast, in this study, one student prepares to teach his partner. Then, they change roles. Thus, during collaboration, their knowledge level differs. Second, in the Rummel et al. study, collaboration was face to face, whereas this study used a chat tool for interaction.

Similar to the adaptive script component of the Collaborative Problem-Solving Script, the [[The Help Tutor Roll Aleven McLaren|Help Tutor project]] aims at improving students' [[help-seeking behavior]] and at reducing students' tendency to [[game the system]]. Furthermore, both studies contain instructions to teach [[metacognition]]. The metacognitive component in our study instructs students to monitor their interaction in order to improve it in subsequent collaborations; the Help Tutor project asks students to evaluate their need for help in order to improve their help-seeking behavior when learning on the Tutor.

Both in this study and in the [[Reflective Dialogues (Katz)|Reflective Dialogue study]] from Katz, students are asked to engage in reflection following each problem-solving. In this study, the reflection concentrates on the collaborative skills, while in Katz' study, the reflection concentrates on students' domain knowledge of the main principles applied in the problem.

Furthermore, both our study and the [[Help Lite (Aleven, Roll)]] aim at improving conceptual knowledge.

=== Future plans ===
Our future plans for June 2007 - December 2007:
* Run full study
* Analyze process data
* Conference contributions at CSCL and AIED
* Paper publication

[[Category:Study]]

Adaptive Assistance for Peer Tutoring (Walker, Rummer, Koedinger)

2008-04-14T19:50:20Z

Erin-Walker: /* Background and Significance */

== Collaborative Extensions to the Cognitive Tutor Algebra: Adaptive Assistance for Peer Tutoring ==
''Erin Walker, Nikol Rummel, and Ken Koedinger''

=== Summary Tables ===
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''PI''' || Erin Walker
|-
| '''Co-PIs''' || Nikol Rummel, Ken Koedinger
|}
 

=== Abstract ===
Our research goal is to integrate a [[collaboration script|peer tutoring script]] within the context of the Cognitive Tutor Algebra (CTA) allowing students to tutor each other through the interface of an [[cognitive tutor|intelligent tutoring system (ITS)]] that provides both domain support and collaborative tutoring. In the PSLC project, “[[Walker A Peer Tutoring Addition|Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition]],” we have added peer tutoring to the Cognitive Tutor Algebra and developed adaptive domain support for the peer tutor. We propose to continue this work by developing and evaluating collaborative assistance for the peer tutor within this context. This assistance will target both the skills required to successfully tutor and the motivation for students to tutor. Once we have shown fixed collaborative assistance to be effective, we plan to implement it in an adaptive fashion, and compare the effects of adaptive and fixed assistance on collaborative skill acquisition and robust domain learning.

The integration of intelligent tutoring and collaborative learning allows us to investigate the differential effects of varying the type and adaptivity of assistance provided to collaborating peers on the acquisition of collaborative skills and on robust domain learning. Further, the development of a successful adaptive collaborative learning system would be a significant contribution to the ITS community.

=== Background and Significance ===

'''Peer Tutoring: Learning by Teaching'''

Incorporating peer tutoring into the CTA might be a way to encourage deep learning. Roscoe and Chi conclude that peer tutors benefit due to knowledge-building, where they reflect on their current knowledge and use it as a basis for constructing new knowledge (Roscoe & Chi, 2007). Because these positive effects are independent of tutor domain ability, researchers implement reciprocal peer tutoring programs, where students of similar abilities take turns tutoring each other. This type of peer tutoring has been shown to increase academic achievement and positive attitudes in long-term classroom interventions (Fantuzzo, Riggio, Connely, & Dimeff, 1989). Biswas et al. (2005) described three properties of peer tutoring related to tutor learning: tutors are accountable for their tutee’s knowledge, they reflect on tutee actions, and they engage in asking questions and giving explanations. Tutee learning is maximized at times when the tutee reaches an impasse, is prompted to find and explain the correct step, and is given an explanation if they fail to do so (VanLehn et al., 2003).

Peer tutors rarely exhibit knowledge-building behaviors spontaneously (Roscoe & Chi, 2007), and thus successful interventions provide them with assistance in order to achieve better learning outcomes for them and their tutees. This assistance can target tutoring behaviors through training, providing positive examples, or structuring the tutoring activities. For example, training students to give conceptual explanations had a significantly positive effect on learning (Fuchs et al., 1997). It is just as critical for assistance to target domain expertise of the peer tutors, in order to ensure that they have sufficient knowledge about a problem to help their partner solve it. Otherwise, there may be cognitive consequences (tutees cannot correctly solve problems) and affective consequences (students feel that they are poor tutors and become discouraged; Medway & Baron, 1997). Domain assistance can take the form of preparation on the problems and scaffolding during tutoring (e.g., Fantuzzo, Riggio, Connely, & Dimeff, 1989). Although assistance for peer tutoring has generally been fixed, providing adaptive support may be a promising approach.

'''Adaptive Collaborative Learning Systems'''

In order to benefit from collaboration students must interact in productive ways, and collaborative activities can be structured (scripted) to encourage these behaviors (e.g., Rummel & Spada, 2007). However, fixed scripts implemented in a one-size-fits-all fashion may be too restrictive for some students and place a high cognitive demand on others (Rummel & Spada, 2007; Dillenbourg, 2002). An adaptive system would be able to monitor student behaviors and provide support only when needed. Preliminary results suggest that adaptive support is indeed beneficial: Adaptive prompting realized in a Wizard of Oz fashion has been shown to have a positive effect on interaction and learning compared to an unscripted condition (Gweon, Rose, Carey, & Zaiss, 2006). An effective way to deliver this support would be to use an adaptive collaborative learning system, where feedback on collaboration is delivered by an intelligent agent.

Work on adaptive collaborative learning systems is still at an early stage. One approach is to use machine learning to detect problematic elements of student interaction in real-time and trigger helpful prompts. Although implementations have lead to significant learning gains, the adaptive feedback appears to be disruptive to dyadic interaction (Kumar et al., 2007). Another promising approach has explored using an intelligent agent as one of the collaborators; students teach the agent about ecosystems with the help of a mentoring agent (Biswas et al. 2005). However, the agents do not interact with the students in natural language, one of the primary benefits of collaboration.

With respect to peer tutoring, intelligent tutoring technology could be applied either to supporting tutor behaviors or domain knowledge of peer tutors. As it is very difficult to build an intelligent tutor for collaborative processes, we decided to develop a general script for the peer tutoring interaction and then focus on providing adaptive domain assistance to peer tutors by leveraging the existing domain models of the CTA. A condition where students tutor each other with adaptive domain support provided to the peer tutor is likely to be better than a condition where the peer tutor merely has access to an answer key, because the support would be tailored to each individual tutor’s needs. It is also likely to be better than a condition where students use the CTA individually, because the students in the collaborative condition would be able to interact deeply about the domain material.

=== Glossary ===
See [[:Category:Peer Tutoring|Peer Tutoring Glossary]]

=== Research Question ===
What are the differential effects of adaptive and fixed support on student collaborative process during a peer tutoring activity and the resulting [[robust learning]] outcomes?

How does an [[instructional method]] that provides metacognitive support and incentives for peer tutoring affect student collaborative process and [[robust learning]] outcomes?

=== Independent variables ===
We vary the [[agent]]s involved in the interaction using the [[instructional method|mode of instruction]]. For example, students may interact individually with a [[cognitive tutor]], may interact with each other using a [[collaboration script]], or may interact with each other with the help of a [[cognitive tutor]].

We also plan to examine the type of assistance provided to the students. They may receive assistance to tutoring competence (either domain, metacognitive, or procedural) or motivational support.

=== Hypotheses ===
1. Students that show high tutoring competence behaviors will show
more domain learning than students that show low tutoring competence behaviors

2. Student perceptions of their tutoring role and beliefs that they can fill
that role are related to their use of effective tutoring behaviors.

3. Either competence or motivational assistance alone is better than no
assistance at promoting good peer tutoring behaviors. Providing peer tutors with
both competence and motivational assistance will lead them to show better peer
tutoring behaviors than with only competence or motivational assistance.

4. An increase in positive peer tutoring behaviors due to fixed
assistance will lead to an increase in peer tutor robust domain learning

5. An increase in positive peer tutoring behaviors due to fixed assistance
will lead to an increase in peer tutee robust domain learning

6. Adaptive assistance is more effective than fixed assistance at
improving peer tutoring behaviors, which promotes robust domain learning of the
peer tutor and peer tutee.

=== Dependent variables ===
* ''[[Normal post-test]]'': Students are given a post-test immediately after the study on isomorphic problems
* ''Far [[transfer]]'': This paper and pencil test assessed students' understanding of the main mathematical concepts from the learning phase. The transfer items students had to solve tapped the same knowledge components as the problems in instruction, however, the problems where non-isomorphic to those in the instruction, thus demanded students to flexibly apply their knowledge to problems with a new format.
* ''[[Long-term retention]]'': Students will be given a test a month after the immediate posttest.
* ''[[Accelerated future learning]] test'': Student learning on future equation solving units will be measured.

To compare [[collaboration skill]]s of students, we will be conducting an analysis of student dialogs during the learning phase.

To assess immediate effects of the instructional variations, we will analyze student progress on training problems as they work through the instruction.

=== Findings ===
We are still in the design phase of this project.

=== Annotated bibliography ===
* Walker, E., Rummel, N., and Koedinger, K. R. To Tutor the Tutor: Adaptive Domain Support for Peer Tutoring. To appear at the 9th International Conference on Intelligent Tutoring Systems. 2008.
* Walker, E., McLaren, B. M., Rummel, N., and Koedinger, K. R. Who Says Three's a Crowd? Using a Cognitive Tutor to Support Peer Tutoring. 13th International Conference on Artificial Intelligence and Education. 2007.
* Walker, E., Rummel, N., McLaren, B. M. & Koedinger, K. R. The Student Becomes the Master: Integrating Peer Tutoring with Cognitive Tutoring. Short paper at the Conference on Computer Supported Collaborative Learning (CSCL-07). Rutgers University, July 16-21, 2007.
* Walker, E., Koedinger, K., McLaren, B. M., & Rummel, N. (2006). Cognitive tutors as research platforms: Extending an established tutoring system for collaborative and metacognitive experimentation. ''Lecture Notes in Computer Science, Volume 4053/2006. Proceedings of the 8th International Conference on Intelligent Tutoring Systems'' (pp. 207-216). Berlin: Springer
* Walker, E. (2005). Mutual peer tutoring: A collaborative addition to the Algebra-1 Cognitive Tutor. Paper presented at the 12th International Conference on Artificial Intelligence and Education (AIED-05, Young Researchers Track), July, 2005, Amsterdam, the Netherlands.

=== References ===
* Roscoe, R. D. & Chi, M. Understanding tutor learning: Knowledge-building and knowledge-telling in peer tutors’ explanations and questions. Review of Educational Research 77(4), 534-574 (2007)
* Fantuzzo, J. W., Riggio, R. E., Connelly, S., & Dimeff, L. A. Effects of reciprocal peer tutoring on academic achievement and psychological adjustment: A component analysis. Journal of Educational Psychology 81(2), 173-177 (1989)
* Biswas, G., Schwartz, D. L., Leelawong, K., Vye, N., & TAG-V. Learning by teaching: A new agent paradigm for educational software. Applied Artificial Intelligence 19, 363–392 (2005)
* VanLehn, K., Siler, S., Murray, C., Yamauchi, T., & Baggett, W. Why do only some events cause learning during human tutoring? Cognition and Instruction 21(3), 209-249 (2003)
* Fuchs, L., Fuchs, D., Hamlett, C., Phillips, N., Karns, K., & Dutka, S. Enhancing students’ helping behaviour during peer-mediated instruction with conceptual mathematical explanations. The Elementary School Journal 97(3), 223-249 (1997)
* Medway, F. & Baron, R. Locus of control and tutors’ instructional style. Contemporary Educational Psychology, 2, 298-310 (1997).
* Rummel, N. & Spada, H. Can people learn computer-mediated collaboration by following a script? In F. Fischer, I. Kollar, H. Mandl &, J. Haake, Scripting computer-supported communication of knowledge. Cognitive, computational, and educational perspectives (pp. 47-63). New York: Springer. (2007)
* Dillenbourg, P. Over-scripting CSCL: The risks of blending collaborative learning with instructional design. In P. A. Kirschner (Ed.), Three worlds of CSCL. Can we support CSCL (pp. 61-91). Heerlen: Open Universiteit Nederland. (2002)
* Gweon, G., Rosé, C., Carey, R. & Zaiss, Z. Providing Support for Adaptive Scripting in an On-Line Collaborative Learning Environment. Proc. of CHI 2006, pp. 251-260. (2006)
* Kumar, R., Rosé, C. P., Wang, Y. C., Joshi, M., Robinson, A. Tutorial dialogue as adaptive collaborative learning support. Proceedings of the 13th International Conference on Artificial Intelligence in Education (AIED 2007), Amsterdam: IOSPress. (2007)

=== Connections ===
This study is an extension of the PSLC project "[[Walker A Peer Tutoring Addition|Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition]]."

Like this study, [[Rummel Scripted Collaborative Problem Solving]] adds scripted collaborative problem solving to the Cognitive Tutor Algebra. The studies differ in the way collaboration is integrated in the Tutor. First, in the Rummel et al. study, both students first prepare one subtasks of a problem to mutually solve the complex story problem later on. Thus, although the students are experts for different parts of the problem, they have a comparable knowledge level during collaboration. In contrast, in this study, one student prepares to teach his partner. Then, they change roles. Thus, during collaboration, their knowledge level differs. Second, in the Rummel et al. study, collaboration was face to face, whereas this study used a chat tool for interaction.

Similar to the adaptive script component of the Collaborative Problem-Solving Script, the [[The Help Tutor Roll Aleven McLaren|Help Tutor project]] aims at improving students' [[help-seeking behavior]] and at reducing students' tendency to [[game the system]]. Furthermore, both studies contain instructions to teach [[metacognition]]. The metacognitive component in our study instructs students to monitor their interaction in order to improve it in subsequent collaborations; the Help Tutor project asks students to evaluate their need for help in order to improve their help-seeking behavior when learning on the Tutor.

Both in this study and in the [[Reflective Dialogues (Katz)|Reflective Dialogue study]] from Katz, students are asked to engage in reflection following each problem-solving. In this study, the reflection concentrates on the collaborative skills, while in Katz' study, the reflection concentrates on students' domain knowledge of the main principles applied in the problem.

Furthermore, both our study and the [[Help Lite (Aleven, Roll)]] aim at improving conceptual knowledge.

=== Future plans ===
Our future plans for June 2008 - December 2008:
* Design & implement further assistance for peer tutoring
* Run lab study evaluating the assistance

[[Category:Study]]

Adaptive Assistance for Peer Tutoring (Walker, Rummer, Koedinger)

2008-04-14T19:49:05Z

Erin-Walker: /* References */

== Collaborative Extensions to the Cognitive Tutor Algebra: Adaptive Assistance for Peer Tutoring ==
''Erin Walker, Nikol Rummel, and Ken Koedinger''

=== Summary Tables ===
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''PI''' || Erin Walker
|-
| '''Co-PIs''' || Nikol Rummel, Ken Koedinger
|}
 

=== Abstract ===
Our research goal is to integrate a [[collaboration script|peer tutoring script]] within the context of the Cognitive Tutor Algebra (CTA) allowing students to tutor each other through the interface of an [[cognitive tutor|intelligent tutoring system (ITS)]] that provides both domain support and collaborative tutoring. In the PSLC project, “[[Walker A Peer Tutoring Addition|Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition]],” we have added peer tutoring to the Cognitive Tutor Algebra and developed adaptive domain support for the peer tutor. We propose to continue this work by developing and evaluating collaborative assistance for the peer tutor within this context. This assistance will target both the skills required to successfully tutor and the motivation for students to tutor. Once we have shown fixed collaborative assistance to be effective, we plan to implement it in an adaptive fashion, and compare the effects of adaptive and fixed assistance on collaborative skill acquisition and robust domain learning.

The integration of intelligent tutoring and collaborative learning allows us to investigate the differential effects of varying the type and adaptivity of assistance provided to collaborating peers on the acquisition of collaborative skills and on robust domain learning. Further, the development of a successful adaptive collaborative learning system would be a significant contribution to the ITS community.

=== Background and Significance ===

'''Peer Tutoring: Learning by Teaching'''

Incorporating peer tutoring into the CTA might be a way to encourage deep learning. Roscoe and Chi conclude that peer tutors benefit due to knowledge-building, where they reflect on their current knowledge and use it as a basis for constructing new knowledge (Roscoe & Chi, 2007). Because these positive effects are independent of tutor domain ability, researchers implement reciprocal peer tutoring programs, where students of similar abilities take turns tutoring each other. This type of peer tutoring has been shown to increase academic achievement and positive attitudes in long-term classroom interventions (Fantuzzo, Riggio, Connely, & Dimeff, 1989). Biswas et al. (2005) described three properties of peer tutoring related to tutor learning: tutors are accountable for their tutee’s knowledge, they reflect on tutee actions, and they engage in asking questions and giving explanations. Tutee learning is maximized at times when the tutee reaches an impasse, is prompted to find and explain the correct step, and is given an explanation if they fail to do so (VanLehn et al., 2003).

Peer tutors rarely exhibit knowledge-building behaviors spontaneously (Roscoe & Chi, 2007), and thus successful interventions provide them with assistance in order to achieve better learning outcomes for them and their tutees. This assistance can target tutoring behaviors through training, providing positive examples, or structuring the tutoring activities. For example, training students to give conceptual explanations had a significantly positive effect on learning (Fuchs et al., 1997). It is just as critical for assistance to target domain expertise of the peer tutors, in order to ensure that they have sufficient knowledge about a problem to help their partner solve it. Otherwise, there may be cognitive consequences (tutees cannot correctly solve problems) and affective consequences (students feel that they are poor tutors and become discouraged; Medway & Baron, 1997). Domain assistance can take the form of preparation on the problems and scaffolding during tutoring (e.g., Fantuzzo, Riggio, Connely, & Dimeff, 1989). Although assistance for peer tutoring has generally been fixed, providing adaptive support may be a promising approach.

'''Adaptive Collaborative Learning Systems'''

In order to benefit from collaboration students must interact in productive ways, and collaborative activities can be structured (scripted) to encourage these behaviors (e.g., Fischer, Kollar, Mandl, & Haake, 2007). However, fixed scripts implemented in a one-size-fits-all fashion may be too restrictive for some students and place a high cognitive demand on others (Fischer et al., 2007; Dillenbourg, 2002). An adaptive system would be able to monitor student behaviors and provide support only when needed. Preliminary results suggest that adaptive support is indeed beneficial: Adaptive prompting realized in a Wizard of Oz fashion has been shown to have a positive effect on interaction and learning compared to an unscripted condition (Gweon, Rose, Carey, & Zaiss, 2006). An effective way to deliver this support would be to use an adaptive collaborative learning system, where feedback on collaboration is delivered by an intelligent agent.

Work on adaptive collaborative learning systems is still at an early stage. One approach is to use machine learning to detect problematic elements of student interaction in real-time and trigger helpful prompts. Although implementations have lead to significant learning gains, the adaptive feedback appears to be disruptive to dyadic interaction (Kumar et al., 2007). Another promising approach has explored using an intelligent agent as one of the collaborators; students teach the agent about ecosystems with the help of a mentoring agent (Biswas et al. 2005). However, the agents do not interact with the students in natural language, one of the primary benefits of collaboration.

With respect to peer tutoring, intelligent tutoring technology could be applied either to supporting tutor behaviors or domain knowledge of peer tutors. As it is very difficult to build an intelligent tutor for collaborative processes, we decided to develop a general script for the peer tutoring interaction and then focus on providing adaptive domain assistance to peer tutors by leveraging the existing domain models of the CTA. A condition where students tutor each other with adaptive domain support provided to the peer tutor is likely to be better than a condition where the peer tutor merely has access to an answer key, because the support would be tailored to each individual tutor’s needs. It is also likely to be better than a condition where students use the CTA individually, because the students in the collaborative condition would be able to interact deeply about the domain material.

=== Glossary ===
See [[:Category:Peer Tutoring|Peer Tutoring Glossary]]

=== Research Question ===
What are the differential effects of adaptive and fixed support on student collaborative process during a peer tutoring activity and the resulting [[robust learning]] outcomes?

How does an [[instructional method]] that provides metacognitive support and incentives for peer tutoring affect student collaborative process and [[robust learning]] outcomes?

=== Independent variables ===
We vary the [[agent]]s involved in the interaction using the [[instructional method|mode of instruction]]. For example, students may interact individually with a [[cognitive tutor]], may interact with each other using a [[collaboration script]], or may interact with each other with the help of a [[cognitive tutor]].

We also plan to examine the type of assistance provided to the students. They may receive assistance to tutoring competence (either domain, metacognitive, or procedural) or motivational support.

=== Hypotheses ===
1. Students that show high tutoring competence behaviors will show
more domain learning than students that show low tutoring competence behaviors

2. Student perceptions of their tutoring role and beliefs that they can fill
that role are related to their use of effective tutoring behaviors.

3. Either competence or motivational assistance alone is better than no
assistance at promoting good peer tutoring behaviors. Providing peer tutors with
both competence and motivational assistance will lead them to show better peer
tutoring behaviors than with only competence or motivational assistance.

4. An increase in positive peer tutoring behaviors due to fixed
assistance will lead to an increase in peer tutor robust domain learning

5. An increase in positive peer tutoring behaviors due to fixed assistance
will lead to an increase in peer tutee robust domain learning

6. Adaptive assistance is more effective than fixed assistance at
improving peer tutoring behaviors, which promotes robust domain learning of the
peer tutor and peer tutee.

=== Dependent variables ===
* ''[[Normal post-test]]'': Students are given a post-test immediately after the study on isomorphic problems
* ''Far [[transfer]]'': This paper and pencil test assessed students' understanding of the main mathematical concepts from the learning phase. The transfer items students had to solve tapped the same knowledge components as the problems in instruction, however, the problems where non-isomorphic to those in the instruction, thus demanded students to flexibly apply their knowledge to problems with a new format.
* ''[[Long-term retention]]'': Students will be given a test a month after the immediate posttest.
* ''[[Accelerated future learning]] test'': Student learning on future equation solving units will be measured.

To compare [[collaboration skill]]s of students, we will be conducting an analysis of student dialogs during the learning phase.

To assess immediate effects of the instructional variations, we will analyze student progress on training problems as they work through the instruction.

=== Findings ===
We are still in the design phase of this project.

=== Annotated bibliography ===
* Walker, E., Rummel, N., and Koedinger, K. R. To Tutor the Tutor: Adaptive Domain Support for Peer Tutoring. To appear at the 9th International Conference on Intelligent Tutoring Systems. 2008.
* Walker, E., McLaren, B. M., Rummel, N., and Koedinger, K. R. Who Says Three's a Crowd? Using a Cognitive Tutor to Support Peer Tutoring. 13th International Conference on Artificial Intelligence and Education. 2007.
* Walker, E., Rummel, N., McLaren, B. M. & Koedinger, K. R. The Student Becomes the Master: Integrating Peer Tutoring with Cognitive Tutoring. Short paper at the Conference on Computer Supported Collaborative Learning (CSCL-07). Rutgers University, July 16-21, 2007.
* Walker, E., Koedinger, K., McLaren, B. M., & Rummel, N. (2006). Cognitive tutors as research platforms: Extending an established tutoring system for collaborative and metacognitive experimentation. ''Lecture Notes in Computer Science, Volume 4053/2006. Proceedings of the 8th International Conference on Intelligent Tutoring Systems'' (pp. 207-216). Berlin: Springer
* Walker, E. (2005). Mutual peer tutoring: A collaborative addition to the Algebra-1 Cognitive Tutor. Paper presented at the 12th International Conference on Artificial Intelligence and Education (AIED-05, Young Researchers Track), July, 2005, Amsterdam, the Netherlands.

=== References ===
* Roscoe, R. D. & Chi, M. Understanding tutor learning: Knowledge-building and knowledge-telling in peer tutors’ explanations and questions. Review of Educational Research 77(4), 534-574 (2007)
* Fantuzzo, J. W., Riggio, R. E., Connelly, S., & Dimeff, L. A. Effects of reciprocal peer tutoring on academic achievement and psychological adjustment: A component analysis. Journal of Educational Psychology 81(2), 173-177 (1989)
* Biswas, G., Schwartz, D. L., Leelawong, K., Vye, N., & TAG-V. Learning by teaching: A new agent paradigm for educational software. Applied Artificial Intelligence 19, 363–392 (2005)
* VanLehn, K., Siler, S., Murray, C., Yamauchi, T., & Baggett, W. Why do only some events cause learning during human tutoring? Cognition and Instruction 21(3), 209-249 (2003)
* Fuchs, L., Fuchs, D., Hamlett, C., Phillips, N., Karns, K., & Dutka, S. Enhancing students’ helping behaviour during peer-mediated instruction with conceptual mathematical explanations. The Elementary School Journal 97(3), 223-249 (1997)
* Medway, F. & Baron, R. Locus of control and tutors’ instructional style. Contemporary Educational Psychology, 2, 298-310 (1997).
* Rummel, N. & Spada, H. Can people learn computer-mediated collaboration by following a script? In F. Fischer, I. Kollar, H. Mandl &, J. Haake, Scripting computer-supported communication of knowledge. Cognitive, computational, and educational perspectives (pp. 47-63). New York: Springer. (2007)
* Dillenbourg, P. Over-scripting CSCL: The risks of blending collaborative learning with instructional design. In P. A. Kirschner (Ed.), Three worlds of CSCL. Can we support CSCL (pp. 61-91). Heerlen: Open Universiteit Nederland. (2002)
* Gweon, G., Rosé, C., Carey, R. & Zaiss, Z. Providing Support for Adaptive Scripting in an On-Line Collaborative Learning Environment. Proc. of CHI 2006, pp. 251-260. (2006)
* Kumar, R., Rosé, C. P., Wang, Y. C., Joshi, M., Robinson, A. Tutorial dialogue as adaptive collaborative learning support. Proceedings of the 13th International Conference on Artificial Intelligence in Education (AIED 2007), Amsterdam: IOSPress. (2007)

=== Connections ===
This study is an extension of the PSLC project "[[Walker A Peer Tutoring Addition|Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition]]."

Like this study, [[Rummel Scripted Collaborative Problem Solving]] adds scripted collaborative problem solving to the Cognitive Tutor Algebra. The studies differ in the way collaboration is integrated in the Tutor. First, in the Rummel et al. study, both students first prepare one subtasks of a problem to mutually solve the complex story problem later on. Thus, although the students are experts for different parts of the problem, they have a comparable knowledge level during collaboration. In contrast, in this study, one student prepares to teach his partner. Then, they change roles. Thus, during collaboration, their knowledge level differs. Second, in the Rummel et al. study, collaboration was face to face, whereas this study used a chat tool for interaction.

Similar to the adaptive script component of the Collaborative Problem-Solving Script, the [[The Help Tutor Roll Aleven McLaren|Help Tutor project]] aims at improving students' [[help-seeking behavior]] and at reducing students' tendency to [[game the system]]. Furthermore, both studies contain instructions to teach [[metacognition]]. The metacognitive component in our study instructs students to monitor their interaction in order to improve it in subsequent collaborations; the Help Tutor project asks students to evaluate their need for help in order to improve their help-seeking behavior when learning on the Tutor.

Both in this study and in the [[Reflective Dialogues (Katz)|Reflective Dialogue study]] from Katz, students are asked to engage in reflection following each problem-solving. In this study, the reflection concentrates on the collaborative skills, while in Katz' study, the reflection concentrates on students' domain knowledge of the main principles applied in the problem.

Furthermore, both our study and the [[Help Lite (Aleven, Roll)]] aim at improving conceptual knowledge.

=== Future plans ===
Our future plans for June 2008 - December 2008:
* Design & implement further assistance for peer tutoring
* Run lab study evaluating the assistance

[[Category:Study]]

Adaptive Assistance for Peer Tutoring (Walker, Rummer, Koedinger)

2008-04-14T19:44:49Z

Erin-Walker: /* Hypotheses */

== Collaborative Extensions to the Cognitive Tutor Algebra: Adaptive Assistance for Peer Tutoring ==
''Erin Walker, Nikol Rummel, and Ken Koedinger''

=== Summary Tables ===
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''PI''' || Erin Walker
|-
| '''Co-PIs''' || Nikol Rummel, Ken Koedinger
|}
 

=== Abstract ===
Our research goal is to integrate a [[collaboration script|peer tutoring script]] within the context of the Cognitive Tutor Algebra (CTA) allowing students to tutor each other through the interface of an [[cognitive tutor|intelligent tutoring system (ITS)]] that provides both domain support and collaborative tutoring. In the PSLC project, “[[Walker A Peer Tutoring Addition|Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition]],” we have added peer tutoring to the Cognitive Tutor Algebra and developed adaptive domain support for the peer tutor. We propose to continue this work by developing and evaluating collaborative assistance for the peer tutor within this context. This assistance will target both the skills required to successfully tutor and the motivation for students to tutor. Once we have shown fixed collaborative assistance to be effective, we plan to implement it in an adaptive fashion, and compare the effects of adaptive and fixed assistance on collaborative skill acquisition and robust domain learning.

The integration of intelligent tutoring and collaborative learning allows us to investigate the differential effects of varying the type and adaptivity of assistance provided to collaborating peers on the acquisition of collaborative skills and on robust domain learning. Further, the development of a successful adaptive collaborative learning system would be a significant contribution to the ITS community.

=== Background and Significance ===

'''Peer Tutoring: Learning by Teaching'''

Incorporating peer tutoring into the CTA might be a way to encourage deep learning. Roscoe and Chi conclude that peer tutors benefit due to knowledge-building, where they reflect on their current knowledge and use it as a basis for constructing new knowledge (Roscoe & Chi, 2007). Because these positive effects are independent of tutor domain ability, researchers implement reciprocal peer tutoring programs, where students of similar abilities take turns tutoring each other. This type of peer tutoring has been shown to increase academic achievement and positive attitudes in long-term classroom interventions (Fantuzzo, Riggio, Connely, & Dimeff, 1989). Biswas et al. (2005) described three properties of peer tutoring related to tutor learning: tutors are accountable for their tutee’s knowledge, they reflect on tutee actions, and they engage in asking questions and giving explanations. Tutee learning is maximized at times when the tutee reaches an impasse, is prompted to find and explain the correct step, and is given an explanation if they fail to do so (VanLehn et al., 2003).

Peer tutors rarely exhibit knowledge-building behaviors spontaneously (Roscoe & Chi, 2007), and thus successful interventions provide them with assistance in order to achieve better learning outcomes for them and their tutees. This assistance can target tutoring behaviors through training, providing positive examples, or structuring the tutoring activities. For example, training students to give conceptual explanations had a significantly positive effect on learning (Fuchs et al., 1997). It is just as critical for assistance to target domain expertise of the peer tutors, in order to ensure that they have sufficient knowledge about a problem to help their partner solve it. Otherwise, there may be cognitive consequences (tutees cannot correctly solve problems) and affective consequences (students feel that they are poor tutors and become discouraged; Medway & Baron, 1997). Domain assistance can take the form of preparation on the problems and scaffolding during tutoring (e.g., Fantuzzo, Riggio, Connely, & Dimeff, 1989). Although assistance for peer tutoring has generally been fixed, providing adaptive support may be a promising approach.

'''Adaptive Collaborative Learning Systems'''

In order to benefit from collaboration students must interact in productive ways, and collaborative activities can be structured (scripted) to encourage these behaviors (e.g., Fischer, Kollar, Mandl, & Haake, 2007). However, fixed scripts implemented in a one-size-fits-all fashion may be too restrictive for some students and place a high cognitive demand on others (Fischer et al., 2007; Dillenbourg, 2002). An adaptive system would be able to monitor student behaviors and provide support only when needed. Preliminary results suggest that adaptive support is indeed beneficial: Adaptive prompting realized in a Wizard of Oz fashion has been shown to have a positive effect on interaction and learning compared to an unscripted condition (Gweon, Rose, Carey, & Zaiss, 2006). An effective way to deliver this support would be to use an adaptive collaborative learning system, where feedback on collaboration is delivered by an intelligent agent.

Work on adaptive collaborative learning systems is still at an early stage. One approach is to use machine learning to detect problematic elements of student interaction in real-time and trigger helpful prompts. Although implementations have lead to significant learning gains, the adaptive feedback appears to be disruptive to dyadic interaction (Kumar et al., 2007). Another promising approach has explored using an intelligent agent as one of the collaborators; students teach the agent about ecosystems with the help of a mentoring agent (Biswas et al. 2005). However, the agents do not interact with the students in natural language, one of the primary benefits of collaboration.

With respect to peer tutoring, intelligent tutoring technology could be applied either to supporting tutor behaviors or domain knowledge of peer tutors. As it is very difficult to build an intelligent tutor for collaborative processes, we decided to develop a general script for the peer tutoring interaction and then focus on providing adaptive domain assistance to peer tutors by leveraging the existing domain models of the CTA. A condition where students tutor each other with adaptive domain support provided to the peer tutor is likely to be better than a condition where the peer tutor merely has access to an answer key, because the support would be tailored to each individual tutor’s needs. It is also likely to be better than a condition where students use the CTA individually, because the students in the collaborative condition would be able to interact deeply about the domain material.

=== Glossary ===
See [[:Category:Peer Tutoring|Peer Tutoring Glossary]]

=== Research Question ===
What are the differential effects of adaptive and fixed support on student collaborative process during a peer tutoring activity and the resulting [[robust learning]] outcomes?

How does an [[instructional method]] that provides metacognitive support and incentives for peer tutoring affect student collaborative process and [[robust learning]] outcomes?

=== Independent variables ===
We vary the [[agent]]s involved in the interaction using the [[instructional method|mode of instruction]]. For example, students may interact individually with a [[cognitive tutor]], may interact with each other using a [[collaboration script]], or may interact with each other with the help of a [[cognitive tutor]].

We also plan to examine the type of assistance provided to the students. They may receive assistance to tutoring competence (either domain, metacognitive, or procedural) or motivational support.

=== Hypotheses ===
1. Students that show high tutoring competence behaviors will show
more domain learning than students that show low tutoring competence behaviors

2. Student perceptions of their tutoring role and beliefs that they can fill
that role are related to their use of effective tutoring behaviors.

3. Either competence or motivational assistance alone is better than no
assistance at promoting good peer tutoring behaviors. Providing peer tutors with
both competence and motivational assistance will lead them to show better peer
tutoring behaviors than with only competence or motivational assistance.

4. An increase in positive peer tutoring behaviors due to fixed
assistance will lead to an increase in peer tutor robust domain learning

5. An increase in positive peer tutoring behaviors due to fixed assistance
will lead to an increase in peer tutee robust domain learning

6. Adaptive assistance is more effective than fixed assistance at
improving peer tutoring behaviors, which promotes robust domain learning of the
peer tutor and peer tutee.

=== Dependent variables ===
* ''[[Normal post-test]]'': Students are given a post-test immediately after the study on isomorphic problems
* ''Far [[transfer]]'': This paper and pencil test assessed students' understanding of the main mathematical concepts from the learning phase. The transfer items students had to solve tapped the same knowledge components as the problems in instruction, however, the problems where non-isomorphic to those in the instruction, thus demanded students to flexibly apply their knowledge to problems with a new format.
* ''[[Long-term retention]]'': Students will be given a test a month after the immediate posttest.
* ''[[Accelerated future learning]] test'': Student learning on future equation solving units will be measured.

To compare [[collaboration skill]]s of students, we will be conducting an analysis of student dialogs during the learning phase.

To assess immediate effects of the instructional variations, we will analyze student progress on training problems as they work through the instruction.

=== Findings ===
We are still in the design phase of this project.

=== Annotated bibliography ===
* Walker, E., Rummel, N., and Koedinger, K. R. To Tutor the Tutor: Adaptive Domain Support for Peer Tutoring. To appear at the 9th International Conference on Intelligent Tutoring Systems. 2008.
* Walker, E., McLaren, B. M., Rummel, N., and Koedinger, K. R. Who Says Three's a Crowd? Using a Cognitive Tutor to Support Peer Tutoring. 13th International Conference on Artificial Intelligence and Education. 2007.
* Walker, E., Rummel, N., McLaren, B. M. & Koedinger, K. R. The Student Becomes the Master: Integrating Peer Tutoring with Cognitive Tutoring. Short paper at the Conference on Computer Supported Collaborative Learning (CSCL-07). Rutgers University, July 16-21, 2007.
* Walker, E., Koedinger, K., McLaren, B. M., & Rummel, N. (2006). Cognitive tutors as research platforms: Extending an established tutoring system for collaborative and metacognitive experimentation. ''Lecture Notes in Computer Science, Volume 4053/2006. Proceedings of the 8th International Conference on Intelligent Tutoring Systems'' (pp. 207-216). Berlin: Springer
* Walker, E. (2005). Mutual peer tutoring: A collaborative addition to the Algebra-1 Cognitive Tutor. Paper presented at the 12th International Conference on Artificial Intelligence and Education (AIED-05, Young Researchers Track), July, 2005, Amsterdam, the Netherlands.

=== References ===
* Roscoe, R. D. & Chi, M. Understanding tutor learning: Knowledge-building and knowledge-telling in peer tutors’ explanations and questions. Review of Educational Research 77(4), 534-574 (2007)
* Fantuzzo, J. W., Riggio, R. E., Connelly, S., & Dimeff, L. A. Effects of reciprocal peer tutoring on academic achievement and psychological adjustment: A component analysis. Journal of Educational Psychology 81(2), 173-177 (1989)
* Biswas, G., Schwartz, D. L., Leelawong, K., Vye, N., & TAG-V. Learning by teaching: A new agent paradigm for educational software. Applied Artificial Intelligence 19, 363–392 (2005)
* VanLehn, K., Siler, S., Murray, C., Yamauchi, T., & Baggett, W. Why do only some events cause learning during human tutoring? Cognition and Instruction 21(3), 209-249 (2003)
* Fuchs, L., Fuchs, D., Hamlett, C., Phillips, N., Karns, K., & Dutka, S. Enhancing students’ helping behaviour during peer-mediated instruction with conceptual mathematical explanations. The Elementary School Journal 97(3), 223-249 (1997)
* Medway, F. & Baron, R. Locus of control and tutors’ instructional style. Contemporary Educational Psychology, 2, 298-310 (1997).
13. Rummel, N. & Spada, H. Can people learn computer-mediated collaboration by following a script? In * F. Fischer, I. Kollar, H. Mandl &, J. Haake, Scripting computer-supported communication of knowledge. Cognitive, computational, and educational perspectives (pp. 47-63). New York: Springer. (2007)
* Dillenbourg, P. Over-scripting CSCL: The risks of blending collaborative learning with instructional design. In P. A. Kirschner (Ed.), Three worlds of CSCL. Can we support CSCL (pp. 61-91). Heerlen: Open Universiteit Nederland. (2002)
* Gweon, G., Rosé, C., Carey, R. & Zaiss, Z. Providing Support for Adaptive Scripting in an On-Line Collaborative Learning Environment. Proc. of CHI 2006, pp. 251-260. (2006)
* Kumar, R., Rosé, C. P., Wang, Y. C., Joshi, M., Robinson, A. Tutorial dialogue as adaptive collaborative learning support. Proceedings of the 13th International Conference on Artificial Intelligence in Education (AIED 2007), Amsterdam: IOSPress. (2007)

=== Connections ===
This study is an extension of the PSLC project "[[Walker A Peer Tutoring Addition|Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition]]."

Like this study, [[Rummel Scripted Collaborative Problem Solving]] adds scripted collaborative problem solving to the Cognitive Tutor Algebra. The studies differ in the way collaboration is integrated in the Tutor. First, in the Rummel et al. study, both students first prepare one subtasks of a problem to mutually solve the complex story problem later on. Thus, although the students are experts for different parts of the problem, they have a comparable knowledge level during collaboration. In contrast, in this study, one student prepares to teach his partner. Then, they change roles. Thus, during collaboration, their knowledge level differs. Second, in the Rummel et al. study, collaboration was face to face, whereas this study used a chat tool for interaction.

Similar to the adaptive script component of the Collaborative Problem-Solving Script, the [[The Help Tutor Roll Aleven McLaren|Help Tutor project]] aims at improving students' [[help-seeking behavior]] and at reducing students' tendency to [[game the system]]. Furthermore, both studies contain instructions to teach [[metacognition]]. The metacognitive component in our study instructs students to monitor their interaction in order to improve it in subsequent collaborations; the Help Tutor project asks students to evaluate their need for help in order to improve their help-seeking behavior when learning on the Tutor.

Both in this study and in the [[Reflective Dialogues (Katz)|Reflective Dialogue study]] from Katz, students are asked to engage in reflection following each problem-solving. In this study, the reflection concentrates on the collaborative skills, while in Katz' study, the reflection concentrates on students' domain knowledge of the main principles applied in the problem.

Furthermore, both our study and the [[Help Lite (Aleven, Roll)]] aim at improving conceptual knowledge.

=== Future plans ===
Our future plans for June 2008 - December 2008:
* Design & implement further assistance for peer tutoring
* Run lab study evaluating the assistance

[[Category:Study]]

Adaptive Assistance for Peer Tutoring (Walker, Rummer, Koedinger)

2008-04-14T19:44:28Z

Erin-Walker: /* Hypotheses */

== Collaborative Extensions to the Cognitive Tutor Algebra: Adaptive Assistance for Peer Tutoring ==
''Erin Walker, Nikol Rummel, and Ken Koedinger''

=== Summary Tables ===
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''PI''' || Erin Walker
|-
| '''Co-PIs''' || Nikol Rummel, Ken Koedinger
|}
 

=== Abstract ===
Our research goal is to integrate a [[collaboration script|peer tutoring script]] within the context of the Cognitive Tutor Algebra (CTA) allowing students to tutor each other through the interface of an [[cognitive tutor|intelligent tutoring system (ITS)]] that provides both domain support and collaborative tutoring. In the PSLC project, “[[Walker A Peer Tutoring Addition|Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition]],” we have added peer tutoring to the Cognitive Tutor Algebra and developed adaptive domain support for the peer tutor. We propose to continue this work by developing and evaluating collaborative assistance for the peer tutor within this context. This assistance will target both the skills required to successfully tutor and the motivation for students to tutor. Once we have shown fixed collaborative assistance to be effective, we plan to implement it in an adaptive fashion, and compare the effects of adaptive and fixed assistance on collaborative skill acquisition and robust domain learning.

The integration of intelligent tutoring and collaborative learning allows us to investigate the differential effects of varying the type and adaptivity of assistance provided to collaborating peers on the acquisition of collaborative skills and on robust domain learning. Further, the development of a successful adaptive collaborative learning system would be a significant contribution to the ITS community.

=== Background and Significance ===

'''Peer Tutoring: Learning by Teaching'''

Incorporating peer tutoring into the CTA might be a way to encourage deep learning. Roscoe and Chi conclude that peer tutors benefit due to knowledge-building, where they reflect on their current knowledge and use it as a basis for constructing new knowledge (Roscoe & Chi, 2007). Because these positive effects are independent of tutor domain ability, researchers implement reciprocal peer tutoring programs, where students of similar abilities take turns tutoring each other. This type of peer tutoring has been shown to increase academic achievement and positive attitudes in long-term classroom interventions (Fantuzzo, Riggio, Connely, & Dimeff, 1989). Biswas et al. (2005) described three properties of peer tutoring related to tutor learning: tutors are accountable for their tutee’s knowledge, they reflect on tutee actions, and they engage in asking questions and giving explanations. Tutee learning is maximized at times when the tutee reaches an impasse, is prompted to find and explain the correct step, and is given an explanation if they fail to do so (VanLehn et al., 2003).

Peer tutors rarely exhibit knowledge-building behaviors spontaneously (Roscoe & Chi, 2007), and thus successful interventions provide them with assistance in order to achieve better learning outcomes for them and their tutees. This assistance can target tutoring behaviors through training, providing positive examples, or structuring the tutoring activities. For example, training students to give conceptual explanations had a significantly positive effect on learning (Fuchs et al., 1997). It is just as critical for assistance to target domain expertise of the peer tutors, in order to ensure that they have sufficient knowledge about a problem to help their partner solve it. Otherwise, there may be cognitive consequences (tutees cannot correctly solve problems) and affective consequences (students feel that they are poor tutors and become discouraged; Medway & Baron, 1997). Domain assistance can take the form of preparation on the problems and scaffolding during tutoring (e.g., Fantuzzo, Riggio, Connely, & Dimeff, 1989). Although assistance for peer tutoring has generally been fixed, providing adaptive support may be a promising approach.

'''Adaptive Collaborative Learning Systems'''

In order to benefit from collaboration students must interact in productive ways, and collaborative activities can be structured (scripted) to encourage these behaviors (e.g., Fischer, Kollar, Mandl, & Haake, 2007). However, fixed scripts implemented in a one-size-fits-all fashion may be too restrictive for some students and place a high cognitive demand on others (Fischer et al., 2007; Dillenbourg, 2002). An adaptive system would be able to monitor student behaviors and provide support only when needed. Preliminary results suggest that adaptive support is indeed beneficial: Adaptive prompting realized in a Wizard of Oz fashion has been shown to have a positive effect on interaction and learning compared to an unscripted condition (Gweon, Rose, Carey, & Zaiss, 2006). An effective way to deliver this support would be to use an adaptive collaborative learning system, where feedback on collaboration is delivered by an intelligent agent.

Work on adaptive collaborative learning systems is still at an early stage. One approach is to use machine learning to detect problematic elements of student interaction in real-time and trigger helpful prompts. Although implementations have lead to significant learning gains, the adaptive feedback appears to be disruptive to dyadic interaction (Kumar et al., 2007). Another promising approach has explored using an intelligent agent as one of the collaborators; students teach the agent about ecosystems with the help of a mentoring agent (Biswas et al. 2005). However, the agents do not interact with the students in natural language, one of the primary benefits of collaboration.

With respect to peer tutoring, intelligent tutoring technology could be applied either to supporting tutor behaviors or domain knowledge of peer tutors. As it is very difficult to build an intelligent tutor for collaborative processes, we decided to develop a general script for the peer tutoring interaction and then focus on providing adaptive domain assistance to peer tutors by leveraging the existing domain models of the CTA. A condition where students tutor each other with adaptive domain support provided to the peer tutor is likely to be better than a condition where the peer tutor merely has access to an answer key, because the support would be tailored to each individual tutor’s needs. It is also likely to be better than a condition where students use the CTA individually, because the students in the collaborative condition would be able to interact deeply about the domain material.

=== Glossary ===
See [[:Category:Peer Tutoring|Peer Tutoring Glossary]]

=== Research Question ===
What are the differential effects of adaptive and fixed support on student collaborative process during a peer tutoring activity and the resulting [[robust learning]] outcomes?

How does an [[instructional method]] that provides metacognitive support and incentives for peer tutoring affect student collaborative process and [[robust learning]] outcomes?

=== Independent variables ===
We vary the [[agent]]s involved in the interaction using the [[instructional method|mode of instruction]]. For example, students may interact individually with a [[cognitive tutor]], may interact with each other using a [[collaboration script]], or may interact with each other with the help of a [[cognitive tutor]].

We also plan to examine the type of assistance provided to the students. They may receive assistance to tutoring competence (either domain, metacognitive, or procedural) or motivational support.

=== Hypotheses ===
1. Students that show high tutoring competence behaviors will show
more domain learning than students that show low tutoring competence behaviors
2. Student perceptions of their tutoring role and beliefs that they can fill
that role are related to their use of effective tutoring behaviors.
3. Either competence or motivational assistance alone is better than no
assistance at promoting good peer tutoring behaviors. Providing peer tutors with
both competence and motivational assistance will lead them to show better peer
tutoring behaviors than with only competence or motivational assistance.
4. An increase in positive peer tutoring behaviors due to fixed
assistance will lead to an increase in peer tutor robust domain learning
5. An increase in positive peer tutoring behaviors due to fixed assistance
will lead to an increase in peer tutee robust domain learning
6. Adaptive assistance is more effective than fixed assistance at
improving peer tutoring behaviors, which promotes robust domain learning of the
peer tutor and peer tutee.

=== Dependent variables ===
* ''[[Normal post-test]]'': Students are given a post-test immediately after the study on isomorphic problems
* ''Far [[transfer]]'': This paper and pencil test assessed students' understanding of the main mathematical concepts from the learning phase. The transfer items students had to solve tapped the same knowledge components as the problems in instruction, however, the problems where non-isomorphic to those in the instruction, thus demanded students to flexibly apply their knowledge to problems with a new format.
* ''[[Long-term retention]]'': Students will be given a test a month after the immediate posttest.
* ''[[Accelerated future learning]] test'': Student learning on future equation solving units will be measured.

To compare [[collaboration skill]]s of students, we will be conducting an analysis of student dialogs during the learning phase.

To assess immediate effects of the instructional variations, we will analyze student progress on training problems as they work through the instruction.

=== Findings ===
We are still in the design phase of this project.

=== Annotated bibliography ===
* Walker, E., Rummel, N., and Koedinger, K. R. To Tutor the Tutor: Adaptive Domain Support for Peer Tutoring. To appear at the 9th International Conference on Intelligent Tutoring Systems. 2008.
* Walker, E., McLaren, B. M., Rummel, N., and Koedinger, K. R. Who Says Three's a Crowd? Using a Cognitive Tutor to Support Peer Tutoring. 13th International Conference on Artificial Intelligence and Education. 2007.
* Walker, E., Rummel, N., McLaren, B. M. & Koedinger, K. R. The Student Becomes the Master: Integrating Peer Tutoring with Cognitive Tutoring. Short paper at the Conference on Computer Supported Collaborative Learning (CSCL-07). Rutgers University, July 16-21, 2007.
* Walker, E., Koedinger, K., McLaren, B. M., & Rummel, N. (2006). Cognitive tutors as research platforms: Extending an established tutoring system for collaborative and metacognitive experimentation. ''Lecture Notes in Computer Science, Volume 4053/2006. Proceedings of the 8th International Conference on Intelligent Tutoring Systems'' (pp. 207-216). Berlin: Springer
* Walker, E. (2005). Mutual peer tutoring: A collaborative addition to the Algebra-1 Cognitive Tutor. Paper presented at the 12th International Conference on Artificial Intelligence and Education (AIED-05, Young Researchers Track), July, 2005, Amsterdam, the Netherlands.

=== References ===
* Roscoe, R. D. & Chi, M. Understanding tutor learning: Knowledge-building and knowledge-telling in peer tutors’ explanations and questions. Review of Educational Research 77(4), 534-574 (2007)
* Fantuzzo, J. W., Riggio, R. E., Connelly, S., & Dimeff, L. A. Effects of reciprocal peer tutoring on academic achievement and psychological adjustment: A component analysis. Journal of Educational Psychology 81(2), 173-177 (1989)
* Biswas, G., Schwartz, D. L., Leelawong, K., Vye, N., & TAG-V. Learning by teaching: A new agent paradigm for educational software. Applied Artificial Intelligence 19, 363–392 (2005)
* VanLehn, K., Siler, S., Murray, C., Yamauchi, T., & Baggett, W. Why do only some events cause learning during human tutoring? Cognition and Instruction 21(3), 209-249 (2003)
* Fuchs, L., Fuchs, D., Hamlett, C., Phillips, N., Karns, K., & Dutka, S. Enhancing students’ helping behaviour during peer-mediated instruction with conceptual mathematical explanations. The Elementary School Journal 97(3), 223-249 (1997)
* Medway, F. & Baron, R. Locus of control and tutors’ instructional style. Contemporary Educational Psychology, 2, 298-310 (1997).
13. Rummel, N. & Spada, H. Can people learn computer-mediated collaboration by following a script? In * F. Fischer, I. Kollar, H. Mandl &, J. Haake, Scripting computer-supported communication of knowledge. Cognitive, computational, and educational perspectives (pp. 47-63). New York: Springer. (2007)
* Dillenbourg, P. Over-scripting CSCL: The risks of blending collaborative learning with instructional design. In P. A. Kirschner (Ed.), Three worlds of CSCL. Can we support CSCL (pp. 61-91). Heerlen: Open Universiteit Nederland. (2002)
* Gweon, G., Rosé, C., Carey, R. & Zaiss, Z. Providing Support for Adaptive Scripting in an On-Line Collaborative Learning Environment. Proc. of CHI 2006, pp. 251-260. (2006)
* Kumar, R., Rosé, C. P., Wang, Y. C., Joshi, M., Robinson, A. Tutorial dialogue as adaptive collaborative learning support. Proceedings of the 13th International Conference on Artificial Intelligence in Education (AIED 2007), Amsterdam: IOSPress. (2007)

=== Connections ===
This study is an extension of the PSLC project "[[Walker A Peer Tutoring Addition|Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition]]."

Like this study, [[Rummel Scripted Collaborative Problem Solving]] adds scripted collaborative problem solving to the Cognitive Tutor Algebra. The studies differ in the way collaboration is integrated in the Tutor. First, in the Rummel et al. study, both students first prepare one subtasks of a problem to mutually solve the complex story problem later on. Thus, although the students are experts for different parts of the problem, they have a comparable knowledge level during collaboration. In contrast, in this study, one student prepares to teach his partner. Then, they change roles. Thus, during collaboration, their knowledge level differs. Second, in the Rummel et al. study, collaboration was face to face, whereas this study used a chat tool for interaction.

Similar to the adaptive script component of the Collaborative Problem-Solving Script, the [[The Help Tutor Roll Aleven McLaren|Help Tutor project]] aims at improving students' [[help-seeking behavior]] and at reducing students' tendency to [[game the system]]. Furthermore, both studies contain instructions to teach [[metacognition]]. The metacognitive component in our study instructs students to monitor their interaction in order to improve it in subsequent collaborations; the Help Tutor project asks students to evaluate their need for help in order to improve their help-seeking behavior when learning on the Tutor.

Both in this study and in the [[Reflective Dialogues (Katz)|Reflective Dialogue study]] from Katz, students are asked to engage in reflection following each problem-solving. In this study, the reflection concentrates on the collaborative skills, while in Katz' study, the reflection concentrates on students' domain knowledge of the main principles applied in the problem.

Furthermore, both our study and the [[Help Lite (Aleven, Roll)]] aim at improving conceptual knowledge.

=== Future plans ===
Our future plans for June 2008 - December 2008:
* Design & implement further assistance for peer tutoring
* Run lab study evaluating the assistance

[[Category:Study]]

Adaptive Assistance for Peer Tutoring (Walker, Rummer, Koedinger)

2008-04-14T19:43:55Z

Erin-Walker: /* Hypotheses */

== Collaborative Extensions to the Cognitive Tutor Algebra: Adaptive Assistance for Peer Tutoring ==
''Erin Walker, Nikol Rummel, and Ken Koedinger''

=== Summary Tables ===
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''PI''' || Erin Walker
|-
| '''Co-PIs''' || Nikol Rummel, Ken Koedinger
|}
 

=== Abstract ===
Our research goal is to integrate a [[collaboration script|peer tutoring script]] within the context of the Cognitive Tutor Algebra (CTA) allowing students to tutor each other through the interface of an [[cognitive tutor|intelligent tutoring system (ITS)]] that provides both domain support and collaborative tutoring. In the PSLC project, “[[Walker A Peer Tutoring Addition|Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition]],” we have added peer tutoring to the Cognitive Tutor Algebra and developed adaptive domain support for the peer tutor. We propose to continue this work by developing and evaluating collaborative assistance for the peer tutor within this context. This assistance will target both the skills required to successfully tutor and the motivation for students to tutor. Once we have shown fixed collaborative assistance to be effective, we plan to implement it in an adaptive fashion, and compare the effects of adaptive and fixed assistance on collaborative skill acquisition and robust domain learning.

The integration of intelligent tutoring and collaborative learning allows us to investigate the differential effects of varying the type and adaptivity of assistance provided to collaborating peers on the acquisition of collaborative skills and on robust domain learning. Further, the development of a successful adaptive collaborative learning system would be a significant contribution to the ITS community.

=== Background and Significance ===

'''Peer Tutoring: Learning by Teaching'''

Incorporating peer tutoring into the CTA might be a way to encourage deep learning. Roscoe and Chi conclude that peer tutors benefit due to knowledge-building, where they reflect on their current knowledge and use it as a basis for constructing new knowledge (Roscoe & Chi, 2007). Because these positive effects are independent of tutor domain ability, researchers implement reciprocal peer tutoring programs, where students of similar abilities take turns tutoring each other. This type of peer tutoring has been shown to increase academic achievement and positive attitudes in long-term classroom interventions (Fantuzzo, Riggio, Connely, & Dimeff, 1989). Biswas et al. (2005) described three properties of peer tutoring related to tutor learning: tutors are accountable for their tutee’s knowledge, they reflect on tutee actions, and they engage in asking questions and giving explanations. Tutee learning is maximized at times when the tutee reaches an impasse, is prompted to find and explain the correct step, and is given an explanation if they fail to do so (VanLehn et al., 2003).

Peer tutors rarely exhibit knowledge-building behaviors spontaneously (Roscoe & Chi, 2007), and thus successful interventions provide them with assistance in order to achieve better learning outcomes for them and their tutees. This assistance can target tutoring behaviors through training, providing positive examples, or structuring the tutoring activities. For example, training students to give conceptual explanations had a significantly positive effect on learning (Fuchs et al., 1997). It is just as critical for assistance to target domain expertise of the peer tutors, in order to ensure that they have sufficient knowledge about a problem to help their partner solve it. Otherwise, there may be cognitive consequences (tutees cannot correctly solve problems) and affective consequences (students feel that they are poor tutors and become discouraged; Medway & Baron, 1997). Domain assistance can take the form of preparation on the problems and scaffolding during tutoring (e.g., Fantuzzo, Riggio, Connely, & Dimeff, 1989). Although assistance for peer tutoring has generally been fixed, providing adaptive support may be a promising approach.

'''Adaptive Collaborative Learning Systems'''

In order to benefit from collaboration students must interact in productive ways, and collaborative activities can be structured (scripted) to encourage these behaviors (e.g., Fischer, Kollar, Mandl, & Haake, 2007). However, fixed scripts implemented in a one-size-fits-all fashion may be too restrictive for some students and place a high cognitive demand on others (Fischer et al., 2007; Dillenbourg, 2002). An adaptive system would be able to monitor student behaviors and provide support only when needed. Preliminary results suggest that adaptive support is indeed beneficial: Adaptive prompting realized in a Wizard of Oz fashion has been shown to have a positive effect on interaction and learning compared to an unscripted condition (Gweon, Rose, Carey, & Zaiss, 2006). An effective way to deliver this support would be to use an adaptive collaborative learning system, where feedback on collaboration is delivered by an intelligent agent.

Work on adaptive collaborative learning systems is still at an early stage. One approach is to use machine learning to detect problematic elements of student interaction in real-time and trigger helpful prompts. Although implementations have lead to significant learning gains, the adaptive feedback appears to be disruptive to dyadic interaction (Kumar et al., 2007). Another promising approach has explored using an intelligent agent as one of the collaborators; students teach the agent about ecosystems with the help of a mentoring agent (Biswas et al. 2005). However, the agents do not interact with the students in natural language, one of the primary benefits of collaboration.

With respect to peer tutoring, intelligent tutoring technology could be applied either to supporting tutor behaviors or domain knowledge of peer tutors. As it is very difficult to build an intelligent tutor for collaborative processes, we decided to develop a general script for the peer tutoring interaction and then focus on providing adaptive domain assistance to peer tutors by leveraging the existing domain models of the CTA. A condition where students tutor each other with adaptive domain support provided to the peer tutor is likely to be better than a condition where the peer tutor merely has access to an answer key, because the support would be tailored to each individual tutor’s needs. It is also likely to be better than a condition where students use the CTA individually, because the students in the collaborative condition would be able to interact deeply about the domain material.

=== Glossary ===
See [[:Category:Peer Tutoring|Peer Tutoring Glossary]]

=== Research Question ===
What are the differential effects of adaptive and fixed support on student collaborative process during a peer tutoring activity and the resulting [[robust learning]] outcomes?

How does an [[instructional method]] that provides metacognitive support and incentives for peer tutoring affect student collaborative process and [[robust learning]] outcomes?

=== Independent variables ===
We vary the [[agent]]s involved in the interaction using the [[instructional method|mode of instruction]]. For example, students may interact individually with a [[cognitive tutor]], may interact with each other using a [[collaboration script]], or may interact with each other with the help of a [[cognitive tutor]].

We also plan to examine the type of assistance provided to the students. They may receive assistance to tutoring competence (either domain, metacognitive, or procedural) or motivational support.

=== Hypotheses ===
1a. Students that show high tutoring competence behaviors will show
more domain learning than students that show low tutoring competence behaviors

1b. Student perceptions of their tutoring role and beliefs that they can fill
that role are related to their use of effective tutoring behaviors.

2a. Either competence or motivational assistance alone is better than no
assistance at promoting good peer tutoring behaviors. Providing peer tutors with
both competence and motivational assistance will lead them to show better peer
tutoring behaviors than with only competence or motivational assistance.

2b. An increase in positive peer tutoring behaviors due to fixed
assistance will lead to an increase in peer tutor robust domain learning

2c. An increase in positive peer tutoring behaviors due to fixed assistance
will lead to an increase in peer tutee robust domain learning

3. Adaptive assistance is more effective than fixed assistance at
improving peer tutoring behaviors, which promotes robust domain learning of the
peer tutor and peer tutee.

=== Dependent variables ===
* ''[[Normal post-test]]'': Students are given a post-test immediately after the study on isomorphic problems
* ''Far [[transfer]]'': This paper and pencil test assessed students' understanding of the main mathematical concepts from the learning phase. The transfer items students had to solve tapped the same knowledge components as the problems in instruction, however, the problems where non-isomorphic to those in the instruction, thus demanded students to flexibly apply their knowledge to problems with a new format.
* ''[[Long-term retention]]'': Students will be given a test a month after the immediate posttest.
* ''[[Accelerated future learning]] test'': Student learning on future equation solving units will be measured.

To compare [[collaboration skill]]s of students, we will be conducting an analysis of student dialogs during the learning phase.

To assess immediate effects of the instructional variations, we will analyze student progress on training problems as they work through the instruction.

=== Findings ===
We are still in the design phase of this project.

=== Annotated bibliography ===
* Walker, E., Rummel, N., and Koedinger, K. R. To Tutor the Tutor: Adaptive Domain Support for Peer Tutoring. To appear at the 9th International Conference on Intelligent Tutoring Systems. 2008.
* Walker, E., McLaren, B. M., Rummel, N., and Koedinger, K. R. Who Says Three's a Crowd? Using a Cognitive Tutor to Support Peer Tutoring. 13th International Conference on Artificial Intelligence and Education. 2007.
* Walker, E., Rummel, N., McLaren, B. M. & Koedinger, K. R. The Student Becomes the Master: Integrating Peer Tutoring with Cognitive Tutoring. Short paper at the Conference on Computer Supported Collaborative Learning (CSCL-07). Rutgers University, July 16-21, 2007.
* Walker, E., Koedinger, K., McLaren, B. M., & Rummel, N. (2006). Cognitive tutors as research platforms: Extending an established tutoring system for collaborative and metacognitive experimentation. ''Lecture Notes in Computer Science, Volume 4053/2006. Proceedings of the 8th International Conference on Intelligent Tutoring Systems'' (pp. 207-216). Berlin: Springer
* Walker, E. (2005). Mutual peer tutoring: A collaborative addition to the Algebra-1 Cognitive Tutor. Paper presented at the 12th International Conference on Artificial Intelligence and Education (AIED-05, Young Researchers Track), July, 2005, Amsterdam, the Netherlands.

=== References ===
* Roscoe, R. D. & Chi, M. Understanding tutor learning: Knowledge-building and knowledge-telling in peer tutors’ explanations and questions. Review of Educational Research 77(4), 534-574 (2007)
* Fantuzzo, J. W., Riggio, R. E., Connelly, S., & Dimeff, L. A. Effects of reciprocal peer tutoring on academic achievement and psychological adjustment: A component analysis. Journal of Educational Psychology 81(2), 173-177 (1989)
* Biswas, G., Schwartz, D. L., Leelawong, K., Vye, N., & TAG-V. Learning by teaching: A new agent paradigm for educational software. Applied Artificial Intelligence 19, 363–392 (2005)
* VanLehn, K., Siler, S., Murray, C., Yamauchi, T., & Baggett, W. Why do only some events cause learning during human tutoring? Cognition and Instruction 21(3), 209-249 (2003)
* Fuchs, L., Fuchs, D., Hamlett, C., Phillips, N., Karns, K., & Dutka, S. Enhancing students’ helping behaviour during peer-mediated instruction with conceptual mathematical explanations. The Elementary School Journal 97(3), 223-249 (1997)
* Medway, F. & Baron, R. Locus of control and tutors’ instructional style. Contemporary Educational Psychology, 2, 298-310 (1997).
13. Rummel, N. & Spada, H. Can people learn computer-mediated collaboration by following a script? In * F. Fischer, I. Kollar, H. Mandl &, J. Haake, Scripting computer-supported communication of knowledge. Cognitive, computational, and educational perspectives (pp. 47-63). New York: Springer. (2007)
* Dillenbourg, P. Over-scripting CSCL: The risks of blending collaborative learning with instructional design. In P. A. Kirschner (Ed.), Three worlds of CSCL. Can we support CSCL (pp. 61-91). Heerlen: Open Universiteit Nederland. (2002)
* Gweon, G., Rosé, C., Carey, R. & Zaiss, Z. Providing Support for Adaptive Scripting in an On-Line Collaborative Learning Environment. Proc. of CHI 2006, pp. 251-260. (2006)
* Kumar, R., Rosé, C. P., Wang, Y. C., Joshi, M., Robinson, A. Tutorial dialogue as adaptive collaborative learning support. Proceedings of the 13th International Conference on Artificial Intelligence in Education (AIED 2007), Amsterdam: IOSPress. (2007)

=== Connections ===
This study is an extension of the PSLC project "[[Walker A Peer Tutoring Addition|Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition]]."

Like this study, [[Rummel Scripted Collaborative Problem Solving]] adds scripted collaborative problem solving to the Cognitive Tutor Algebra. The studies differ in the way collaboration is integrated in the Tutor. First, in the Rummel et al. study, both students first prepare one subtasks of a problem to mutually solve the complex story problem later on. Thus, although the students are experts for different parts of the problem, they have a comparable knowledge level during collaboration. In contrast, in this study, one student prepares to teach his partner. Then, they change roles. Thus, during collaboration, their knowledge level differs. Second, in the Rummel et al. study, collaboration was face to face, whereas this study used a chat tool for interaction.

Similar to the adaptive script component of the Collaborative Problem-Solving Script, the [[The Help Tutor Roll Aleven McLaren|Help Tutor project]] aims at improving students' [[help-seeking behavior]] and at reducing students' tendency to [[game the system]]. Furthermore, both studies contain instructions to teach [[metacognition]]. The metacognitive component in our study instructs students to monitor their interaction in order to improve it in subsequent collaborations; the Help Tutor project asks students to evaluate their need for help in order to improve their help-seeking behavior when learning on the Tutor.

Both in this study and in the [[Reflective Dialogues (Katz)|Reflective Dialogue study]] from Katz, students are asked to engage in reflection following each problem-solving. In this study, the reflection concentrates on the collaborative skills, while in Katz' study, the reflection concentrates on students' domain knowledge of the main principles applied in the problem.

Furthermore, both our study and the [[Help Lite (Aleven, Roll)]] aim at improving conceptual knowledge.

=== Future plans ===
Our future plans for June 2008 - December 2008:
* Design & implement further assistance for peer tutoring
* Run lab study evaluating the assistance

[[Category:Study]]

Adaptive Assistance for Peer Tutoring (Walker, Rummer, Koedinger)

2008-04-14T19:42:33Z

Erin-Walker:

== Collaborative Extensions to the Cognitive Tutor Algebra: Adaptive Assistance for Peer Tutoring ==
''Erin Walker, Nikol Rummel, and Ken Koedinger''

=== Summary Tables ===
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''PI''' || Erin Walker
|-
| '''Co-PIs''' || Nikol Rummel, Ken Koedinger
|}
 

=== Abstract ===
Our research goal is to integrate a [[collaboration script|peer tutoring script]] within the context of the Cognitive Tutor Algebra (CTA) allowing students to tutor each other through the interface of an [[cognitive tutor|intelligent tutoring system (ITS)]] that provides both domain support and collaborative tutoring. In the PSLC project, “[[Walker A Peer Tutoring Addition|Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition]],” we have added peer tutoring to the Cognitive Tutor Algebra and developed adaptive domain support for the peer tutor. We propose to continue this work by developing and evaluating collaborative assistance for the peer tutor within this context. This assistance will target both the skills required to successfully tutor and the motivation for students to tutor. Once we have shown fixed collaborative assistance to be effective, we plan to implement it in an adaptive fashion, and compare the effects of adaptive and fixed assistance on collaborative skill acquisition and robust domain learning.

The integration of intelligent tutoring and collaborative learning allows us to investigate the differential effects of varying the type and adaptivity of assistance provided to collaborating peers on the acquisition of collaborative skills and on robust domain learning. Further, the development of a successful adaptive collaborative learning system would be a significant contribution to the ITS community.

=== Background and Significance ===

'''Peer Tutoring: Learning by Teaching'''

Incorporating peer tutoring into the CTA might be a way to encourage deep learning. Roscoe and Chi conclude that peer tutors benefit due to knowledge-building, where they reflect on their current knowledge and use it as a basis for constructing new knowledge (Roscoe & Chi, 2007). Because these positive effects are independent of tutor domain ability, researchers implement reciprocal peer tutoring programs, where students of similar abilities take turns tutoring each other. This type of peer tutoring has been shown to increase academic achievement and positive attitudes in long-term classroom interventions (Fantuzzo, Riggio, Connely, & Dimeff, 1989). Biswas et al. (2005) described three properties of peer tutoring related to tutor learning: tutors are accountable for their tutee’s knowledge, they reflect on tutee actions, and they engage in asking questions and giving explanations. Tutee learning is maximized at times when the tutee reaches an impasse, is prompted to find and explain the correct step, and is given an explanation if they fail to do so (VanLehn et al., 2003).

Peer tutors rarely exhibit knowledge-building behaviors spontaneously (Roscoe & Chi, 2007), and thus successful interventions provide them with assistance in order to achieve better learning outcomes for them and their tutees. This assistance can target tutoring behaviors through training, providing positive examples, or structuring the tutoring activities. For example, training students to give conceptual explanations had a significantly positive effect on learning (Fuchs et al., 1997). It is just as critical for assistance to target domain expertise of the peer tutors, in order to ensure that they have sufficient knowledge about a problem to help their partner solve it. Otherwise, there may be cognitive consequences (tutees cannot correctly solve problems) and affective consequences (students feel that they are poor tutors and become discouraged; Medway & Baron, 1997). Domain assistance can take the form of preparation on the problems and scaffolding during tutoring (e.g., Fantuzzo, Riggio, Connely, & Dimeff, 1989). Although assistance for peer tutoring has generally been fixed, providing adaptive support may be a promising approach.

'''Adaptive Collaborative Learning Systems'''

In order to benefit from collaboration students must interact in productive ways, and collaborative activities can be structured (scripted) to encourage these behaviors (e.g., Fischer, Kollar, Mandl, & Haake, 2007). However, fixed scripts implemented in a one-size-fits-all fashion may be too restrictive for some students and place a high cognitive demand on others (Fischer et al., 2007; Dillenbourg, 2002). An adaptive system would be able to monitor student behaviors and provide support only when needed. Preliminary results suggest that adaptive support is indeed beneficial: Adaptive prompting realized in a Wizard of Oz fashion has been shown to have a positive effect on interaction and learning compared to an unscripted condition (Gweon, Rose, Carey, & Zaiss, 2006). An effective way to deliver this support would be to use an adaptive collaborative learning system, where feedback on collaboration is delivered by an intelligent agent.

Work on adaptive collaborative learning systems is still at an early stage. One approach is to use machine learning to detect problematic elements of student interaction in real-time and trigger helpful prompts. Although implementations have lead to significant learning gains, the adaptive feedback appears to be disruptive to dyadic interaction (Kumar et al., 2007). Another promising approach has explored using an intelligent agent as one of the collaborators; students teach the agent about ecosystems with the help of a mentoring agent (Biswas et al. 2005). However, the agents do not interact with the students in natural language, one of the primary benefits of collaboration.

With respect to peer tutoring, intelligent tutoring technology could be applied either to supporting tutor behaviors or domain knowledge of peer tutors. As it is very difficult to build an intelligent tutor for collaborative processes, we decided to develop a general script for the peer tutoring interaction and then focus on providing adaptive domain assistance to peer tutors by leveraging the existing domain models of the CTA. A condition where students tutor each other with adaptive domain support provided to the peer tutor is likely to be better than a condition where the peer tutor merely has access to an answer key, because the support would be tailored to each individual tutor’s needs. It is also likely to be better than a condition where students use the CTA individually, because the students in the collaborative condition would be able to interact deeply about the domain material.

=== Glossary ===
See [[:Category:Peer Tutoring|Peer Tutoring Glossary]]

=== Research Question ===
What are the differential effects of adaptive and fixed support on student collaborative process during a peer tutoring activity and the resulting [[robust learning]] outcomes?

How does an [[instructional method]] that provides metacognitive support and incentives for peer tutoring affect student collaborative process and [[robust learning]] outcomes?

=== Independent variables ===
We vary the [[agent]]s involved in the interaction using the [[instructional method|mode of instruction]]. For example, students may interact individually with a [[cognitive tutor]], may interact with each other using a [[collaboration script]], or may interact with each other with the help of a [[cognitive tutor]].

We also plan to examine the type of assistance provided to the students. They may receive assistance to tutoring competence (either domain, metacognitive, or procedural) or motivational support.

=== Hypotheses ===
1a. Students that show high tutoring competence behaviors will show
more domain learning than students that show low tutoring competence behaviors
1b. Student perceptions of their tutoring role and beliefs that they can fill
that role are related to their use of effective tutoring behaviors.

2a. Either competence or motivational assistance alone is better than no
assistance at promoting good peer tutoring behaviors. Providing peer tutors with
both competence and motivational assistance will lead them to show better peer
tutoring behaviors than with only competence or motivational assistance.
2b. An increase in positive peer tutoring behaviors due to fixed
assistance will lead to an increase in peer tutor robust domain learning
2c. An increase in positive peer tutoring behaviors due to fixed assistance
will lead to an increase in peer tutee robust domain learning

3. Adaptive assistance is more effective than fixed assistance at
improving peer tutoring behaviors, which promotes robust domain learning of the
peer tutor and peer tutee.

=== Dependent variables ===
* ''[[Normal post-test]]'': Students are given a post-test immediately after the study on isomorphic problems
* ''Far [[transfer]]'': This paper and pencil test assessed students' understanding of the main mathematical concepts from the learning phase. The transfer items students had to solve tapped the same knowledge components as the problems in instruction, however, the problems where non-isomorphic to those in the instruction, thus demanded students to flexibly apply their knowledge to problems with a new format.
* ''[[Long-term retention]]'': Students will be given a test a month after the immediate posttest.
* ''[[Accelerated future learning]] test'': Student learning on future equation solving units will be measured.

To compare [[collaboration skill]]s of students, we will be conducting an analysis of student dialogs during the learning phase.

To assess immediate effects of the instructional variations, we will analyze student progress on training problems as they work through the instruction.

=== Findings ===
We are still in the design phase of this project.

=== Annotated bibliography ===
* Walker, E., Rummel, N., and Koedinger, K. R. To Tutor the Tutor: Adaptive Domain Support for Peer Tutoring. To appear at the 9th International Conference on Intelligent Tutoring Systems. 2008.
* Walker, E., McLaren, B. M., Rummel, N., and Koedinger, K. R. Who Says Three's a Crowd? Using a Cognitive Tutor to Support Peer Tutoring. 13th International Conference on Artificial Intelligence and Education. 2007.
* Walker, E., Rummel, N., McLaren, B. M. & Koedinger, K. R. The Student Becomes the Master: Integrating Peer Tutoring with Cognitive Tutoring. Short paper at the Conference on Computer Supported Collaborative Learning (CSCL-07). Rutgers University, July 16-21, 2007.
* Walker, E., Koedinger, K., McLaren, B. M., & Rummel, N. (2006). Cognitive tutors as research platforms: Extending an established tutoring system for collaborative and metacognitive experimentation. ''Lecture Notes in Computer Science, Volume 4053/2006. Proceedings of the 8th International Conference on Intelligent Tutoring Systems'' (pp. 207-216). Berlin: Springer
* Walker, E. (2005). Mutual peer tutoring: A collaborative addition to the Algebra-1 Cognitive Tutor. Paper presented at the 12th International Conference on Artificial Intelligence and Education (AIED-05, Young Researchers Track), July, 2005, Amsterdam, the Netherlands.

=== References ===
* Roscoe, R. D. & Chi, M. Understanding tutor learning: Knowledge-building and knowledge-telling in peer tutors’ explanations and questions. Review of Educational Research 77(4), 534-574 (2007)
* Fantuzzo, J. W., Riggio, R. E., Connelly, S., & Dimeff, L. A. Effects of reciprocal peer tutoring on academic achievement and psychological adjustment: A component analysis. Journal of Educational Psychology 81(2), 173-177 (1989)
* Biswas, G., Schwartz, D. L., Leelawong, K., Vye, N., & TAG-V. Learning by teaching: A new agent paradigm for educational software. Applied Artificial Intelligence 19, 363–392 (2005)
* VanLehn, K., Siler, S., Murray, C., Yamauchi, T., & Baggett, W. Why do only some events cause learning during human tutoring? Cognition and Instruction 21(3), 209-249 (2003)
* Fuchs, L., Fuchs, D., Hamlett, C., Phillips, N., Karns, K., & Dutka, S. Enhancing students’ helping behaviour during peer-mediated instruction with conceptual mathematical explanations. The Elementary School Journal 97(3), 223-249 (1997)
* Medway, F. & Baron, R. Locus of control and tutors’ instructional style. Contemporary Educational Psychology, 2, 298-310 (1997).
13. Rummel, N. & Spada, H. Can people learn computer-mediated collaboration by following a script? In * F. Fischer, I. Kollar, H. Mandl &, J. Haake, Scripting computer-supported communication of knowledge. Cognitive, computational, and educational perspectives (pp. 47-63). New York: Springer. (2007)
* Dillenbourg, P. Over-scripting CSCL: The risks of blending collaborative learning with instructional design. In P. A. Kirschner (Ed.), Three worlds of CSCL. Can we support CSCL (pp. 61-91). Heerlen: Open Universiteit Nederland. (2002)
* Gweon, G., Rosé, C., Carey, R. & Zaiss, Z. Providing Support for Adaptive Scripting in an On-Line Collaborative Learning Environment. Proc. of CHI 2006, pp. 251-260. (2006)
* Kumar, R., Rosé, C. P., Wang, Y. C., Joshi, M., Robinson, A. Tutorial dialogue as adaptive collaborative learning support. Proceedings of the 13th International Conference on Artificial Intelligence in Education (AIED 2007), Amsterdam: IOSPress. (2007)

=== Connections ===
This study is an extension of the PSLC project "[[Walker A Peer Tutoring Addition|Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition]]."

Like this study, [[Rummel Scripted Collaborative Problem Solving]] adds scripted collaborative problem solving to the Cognitive Tutor Algebra. The studies differ in the way collaboration is integrated in the Tutor. First, in the Rummel et al. study, both students first prepare one subtasks of a problem to mutually solve the complex story problem later on. Thus, although the students are experts for different parts of the problem, they have a comparable knowledge level during collaboration. In contrast, in this study, one student prepares to teach his partner. Then, they change roles. Thus, during collaboration, their knowledge level differs. Second, in the Rummel et al. study, collaboration was face to face, whereas this study used a chat tool for interaction.

Similar to the adaptive script component of the Collaborative Problem-Solving Script, the [[The Help Tutor Roll Aleven McLaren|Help Tutor project]] aims at improving students' [[help-seeking behavior]] and at reducing students' tendency to [[game the system]]. Furthermore, both studies contain instructions to teach [[metacognition]]. The metacognitive component in our study instructs students to monitor their interaction in order to improve it in subsequent collaborations; the Help Tutor project asks students to evaluate their need for help in order to improve their help-seeking behavior when learning on the Tutor.

Both in this study and in the [[Reflective Dialogues (Katz)|Reflective Dialogue study]] from Katz, students are asked to engage in reflection following each problem-solving. In this study, the reflection concentrates on the collaborative skills, while in Katz' study, the reflection concentrates on students' domain knowledge of the main principles applied in the problem.

Furthermore, both our study and the [[Help Lite (Aleven, Roll)]] aim at improving conceptual knowledge.

=== Future plans ===
Our future plans for June 2008 - December 2008:
* Design & implement further assistance for peer tutoring
* Run lab study evaluating the assistance

[[Category:Study]]

Adaptive Assistance for Peer Tutoring (Walker, Rummer, Koedinger)

2008-04-14T19:20:38Z

Erin-Walker:

Adaptive Assistance for Peer Tutoring (Walker, Rummer, Koedinger)

2008-04-14T19:10:35Z

Erin-Walker:

Adaptive Assistance for Peer Tutoring (Walker, Rummer, Koedinger)

2008-04-14T19:04:33Z

Erin-Walker:

Adaptive Assistance for Peer Tutoring (Walker, Rummer, Koedinger)

2008-04-14T19:00:26Z

Walker A Peer Tutoring Addition

2007-03-30T22:29:08Z

Erin-Walker: /* Connections */

== Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition ==
''Erin Walker, Bruce M. McLaren, Nikol Rummel, and Ken Koedinger''

=== Summary Tables ===
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''PIs''' || Bruce McLaren, Nikol Rummel
|-
| '''Other Contributers''' ||
* Graduate Student: Erin Walker
* Staff: Jonathan Steinhart
|}
 
'' Pre Study 1''
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''Study Start Date''' || 05/06
|-
| '''Study End Date''' || 06/06
|-
| '''LearnLab Site''' || CWCTC
|-
| '''LearnLab Course''' || Algebra
|-
| '''Number of Students''' || ''N'' = 14
|-
| '''Average # of hours per participant''' || 3 hrs.
|}
 

'' Pre Study 2''
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''Study Start Date''' || 11/06
|-
| '''Study End Date''' || 11/06
|-
| '''LearnLab Site''' || CWCTC
|-
| '''LearnLab Course''' || Algebra
|-
| '''Number of Students''' || ''N'' = 20
|-
| '''Average # of hours per participant''' || 3 hrs.
|}
 

'' Full Study''
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''Study Start Date''' || 04/07
|-
| '''Study End Date''' || 04/07
|-
| '''LearnLab Site''' || CWCTC
|-
| '''LearnLab Course''' || Algebra
|-
| '''Number of Students''' || ''N'' = 70
|-
| '''Average # of hours per participant''' || 3 hrs.
|}
 

=== Abstract ===
In this project, the Algebra I Cognitive Tutor is extended to a [[collaborative learning environment]]: Instead of the computer tutoring a student, students tutor each other, and the [[cognitive tutor]] provides collaborative support. As research has shown, collaborative problem solving and learning has the potential to increase elaboration on the learning content. However, students are not always able to effectively meet the challenges of a collaborative setting. To ensure that students capitalize on collaborative problem solving with the Tutor, a peer tutoring [[collaboration scripts|collaboration script]] was developed that guides their interaction and prompts fruitful [[collaboration]].

In the baseline version of the script (without cognitive tutor support), students in the same class are seated at different computers and take turs tutoring each other. First, they prepare to tutor by solving problems using the regular cognitive tutor. Then, they tutor each other by marking each other right and wrong, monitoring their partner's skills, and give each other hints and feedback. After preparing and tutoring, they reflect on the problems and their interaction, engaging in [[metacognition|metacognitive activities]]. We have expanded the script to include intelligent cognitive support of the peer tutoring, and hope to ultimately expand it to include intelligent collaborative support.

To assess the effectiveness of the script on [[robust learning]], we are conducting a series of [[in vivo experiment]]s in the Algebra LearnLab. In an initial, small scale study (pre study) that served to establish basic effects and to test the procedure in a classroom setting, we compared peer tutoring with reflection elements to peer tutoring without reflection elements. In a second study that served to explore the effects of cognitive tutoring of peer tutoring, we compared peer tutoring plus cognitive tutoring to peer tutoring. In the full study, we will compare these collaborative conditions to an individual condition to assess the effect of the collaborative Tutor extension to regular Tutor use.

=== Background and Significance ===

In our project, we combined two different [[instructional method]]s both of which have been shown to improve students’ learning in mathematics: Learning with intelligent tutoring systems (Koedinger, Anderson, Hadley, & Mark, 1997) and collaborative problem solving (Berg, 1993). The [[Cognitive tutor|Cognitive Tutor]] Algebra that was used in our study is a tutor for mathematics instruction at the high school level. Students learn with the Tutor during part of the regular classroom sessions. The Tutor's main features are immediate error feedback, the possibility to ask for a hint when encountering impasses, and knowledge tracing, i.e. the Tutor creates and updates a model of the student’s knowledge and selects new problems tailored to the student’s knowledge level. Although several studies have proven its effectiveness, students do not always benefit from learning with the Tutor. First, because the Tutor places emphasis on learning [[procedural|procedural problem solving skills]], yet a deep understanding of underlying mathematical [[conceptual knowledge|concepts]] is not necessarily achieved (Anderson, Corbett, Koedinger, & Pelletier, 1995). Second, students do not always make good use of the learning opportunities provided by the Cognitive Tutor (e.g. [[help abuse]], see Aleven, McLaren, Roll, & Koedinger, 2004; [[game the system|gaming the system]], Baker, Corbett, & Koedinger, 2004). So far, the Cognitive Tutor has been used in an individual learning setting only. However, as research on collaborative learning has shown, collaboration can yield elaboration of learning content (Teasley, 1995), thus this could be a promising approach to reduce the Tutor’s shortcomings. On the other hand, students are not always able to effectively meet the challenges of a collaborative setting (Rummel & Spada, 2005). [[Collaboration scripts]] have proven effective in helping people meet the challenges encountered when learning or working collaboratively (Kollar, Fischer, & Hesse, in press).

Combining collaborative activities with intelligent tutoring might combine the benefits of both approaches. Collaborative interaction could augment the effects of an existing ITS by adding deeper interaction, and intelligent tutoring support could improve the quality of student collaboration by providing guidance to students as they attempt to follow a collaboration script. One way to incorporate collaboration into an ITS is to place the intelligent tutoring system in the role of one of the collaborators and have a student interact with this agent. For example, Biswas, Schwartz, Leelawong, Vye, and the TAG-V (2005) have the student collaborate with an agent (“Betty”) in a peer tutoring scenario. The student teaches Betty about ecosystems with the help of a second mentoring agent (“Mr. Davis”), and it has been shown that a student who takes the role of a teacher in this system benefits more than a student who takes the role of a learner in a parallel traditional tutoring system. However, this system does not yet support natural language interaction between the agents involved, and therefore does not take full advantage of important collaborative interaction mechanisms that can increase learning. Another way to combine collaboration and tutoring is to use the ITS to structure and tutor the student collaboration, by connecting collaborative tools with ITS components (see Harrer, McLaren, Walker, Bollen, & Sewall, 2006), and developing a model of good and ineffective collaboration in order to provide on-line hints and feedback (see Soller, 2001). Most research into computer tutoring of the collaboration between two students is still in a preliminary stage.

We integrate collaborative learning with an ITS using a peer tutoring framework. Similar to Biswas et al. (2005), we put the student in the role of the tutor, and intend to have the cognitive elements of the tutoring supported by another agent. However, unlike Biswas et al., (2005), the teachable agent is a real student. Therefore, we have intelligent tutoring components monitoring and providing feedback on student collaboration in addition to student cognition, like Harrer et al. (2006). Our ultimate goal is to allow students to tutor each other through the interface of an ITS, supported by both cognitive and collaborative tutoring.

=== Glossary ===
See [[:Category:Peer Tutoring|Peer Tutoring Glossary]]

=== Research question ===
Does [[collaboration]] – and in particular [[collaboration scripts|scripted collaboration]] – improve students’ [[robust learning]] in the domain of algebra?

Do [[instructional method|instructional activities]] that involve three agents (a student, a peer tutor, and a computer tutor) increase [[robust learning]] compared to instructional activities that involve two agents (student and peer tutor, student and computer tutor)?

=== Independent variables ===
We vary the [[agent]]s involved in the interaction using the [[instructional method|mode of instruction]]. There are four conditions:
* Individual condition: Students solved problems with the Algebra I Cognitive Tutor in the regular fashion (i.e. computer program as additional [[agent]]).
* Peer tutoring condition: Students tutored each other using the cognitive tutor interface (i.e., another peer as additional [[agent]]). They went through two phases:
** a preparation phase, where students prepared to tutor while solving problems with the Algebra I Cognitive Tutor in the regular fashion
** a collaboration phase, where students take turns tutoring each other. Peer tutors can mark answers right or wrong, monitor their partner's skills, and give hints and feedback
* Peer tutoring plus reflection: As in the peer tutoring condition, students go through a preparation phase and then tutor each other in the collaboration phase. However, students also go through a series of [[metacognition|metacognitive]] self-reflection activities to encourage them to engage in [[elaborative interaction]]
* Peer tutoring plus cognitive tutoring: Students participate in the peer tutoring plus reflection condition, but during the collaboration phase, peer tutors receive cognitive tutor hints and feedback (i.e., a peer and computer program as additional [[agent]]s). This feedback contains two components:
** a collaborative prompt that encourages students to interact with each other
** a cognitive hint targeted at the particular mistake the peer tutee is making

=== Hypothesis ===
1. We expect the collaborative conditions to outperform the individual conditions in measures of robust learning of the algebra content.

2. We epect the peer tutoring plus cognitive tutoring condition to outperform the peer tutoring alone conditions in measures of robust learning of the algebra content

=== Dependent variables ===
* ''Near [[transfer]], immediate'': During training, student progress on training problems are be analyzed.
* ''Near [[transfer]], [[long-term retention|retention]]'': Students a are given a posttest immediately after the study on isomorphic problems
* ''Far [[transfer]]'': This paper and pencil test assessed students' understanding of the main mathematical concepts from the learning phase. The [[conceptual tasks]] students had to solve tapped the same knowledge components as the problems in instruction; however, the problems where non-isomorphic to those in the instruction, thus demanded students to flexibly apply their knowledge to problems with a new format.
* ''Acceleration of future learning'': Student learning on future equation solving units will be measured.

To compare [[collaboration skill]]s of students, we will be conducting an analysis of student dialogues durng the learning phase.

=== Explanation ===
This study is part of the [[Interactive_Communication|Interactive Communication cluster]], and its hypothesis is a specialization of the IC cluster’s central hypothesis. We assume that the instruction is in or above the student’s ZPD. Here is what we expect to happen in the three conditions:

* In the peer tutoring condition, the peer tutor’s help should increase learning compared to a student learning with the cognitive tutor. However, the student may make too many errors and/or require too much communication with the second agent. The peer tutor may not be able to provide the student with the help he/she needs. Learning will not be optimized.
* In the cognitive+peer tutoring condition, the peer tutor’s help should provide the additional benefits of interactive communication, and the cognitive tutor’s help should ensure that both the student and the peer tutor get the problem solving assistance they need. The support of the cognitive tutor and interaction with the student will also increase the peer tutor’s learning.
The target path will have the students learning-by-doing and learning-by-teaching, with the help of the computer agent so students do not make too many errors or require too much communication.

=== Findings ===
'''Pre-study 1:'''

Our first study compared the peer tutoring to peer tutoring plus reflection conditions. Using pretest and posttest scores, we conducted a two-way (condition x test-time) repeated-measure ANOVA, with test-time as the repeated measure. Posttest scores were significantly higher than pretest scores in both the tutoring and the tutoring+reflection condition (F (1,12) = 15.25, p < .002, η² = 0.56), but there were no significant differences between conditions, and no interaction (see Table 2). To further examine what occurred during the collaboration phase we turned to log data and notes from classroom observation. During peer tutoring, students appeared engaged, and did exhibit many of the positive collaborative behaviors that we were attempting to encourage with our script and that have been shown to correlate with knowledge construction and self-reflection. However, we observed that peer tutors struggled to provide tutees with answers, and did not connect the preparation that they had done with the collaboration phase. For instance, they often did not consult their answer printouts when they did not know the next problem step and thus had to rely on teacher assistance to solve a problem. As a result, tutees skipped problems without completing them correctly. This undesirable behavior differed between the two conditions. Students in the tutoring condition attempted more problems than students in the tutoring+reflection condition, and appeared to complete more problems as well. The average number of problems completed by dyads in the tutoring+reflection condition was low; students in this group took an average of 11 minutes to complete a single problem, compared to a 6 minute average in the tutoring condition. Students in the tutoring condition tended to skip problems they could not solve, completing less than 60% of the problems they attempted. Immediately before skipping a problem, students would generally state their inability to solve it, “I don’t know how to do this one,“ or their lack of motivation, “Just do something and I’ll agree or something.” If students skip problems, they may not learn how to solve difficult problems. However, if they do not complete many problems, they may not be sufficiently exposed to all the skills involved in the unit, and will be given fewer opportunities to master them.

'''Results for Pre-study 2 and Study 1 coming soon.'''

=== Explanation ===
This study is part of the [[Interactive_Communication|Interactive Communication cluster]], and its hypothesis is a specialization of the IC cluster’s central hypothesis. According to the IC cluster’s hypothesis, instruction that yields [[robust learning]] should be designed to have the right target paths. Furthermore, it should improve the students’ path choice in the [[learning event space]], i.e., increasing the probability that students take correct paths and decreasing the probability that students take alternative paths.

By enhancing the Cognitive Tutor with collaboration, the study tried to reach these two goals in two steps:

* ''Individual condition'': The existing Cognitive Tutor already offers correct learning paths. However, individual learning on the Cognitive Tutor has several shortcomings. For example, students don’t have the opportunity to reflect on the underlying mathematical [[conceptual knowledge|concepts]] in natural language interaction in order to gain a deeper understanding. In addition, students do not always use the offered learning paths (as the opportunity to reflect on on-demand hints and error feedback) effectively, thus often do not engage in [[sense making]] processes
* ''Peer tutoring condition (1st step)'': By enhancing the Algebra I Cognitive Tutor to be a collaborative learning environment, we added a learning resource – the learning partner. This adds further correct learning paths to the [[learning event space]], for instance, learning by giving explanations, the possibility to request help, and learning by knowledge co-construction. However, similarly to the learning paths in an individual setting, students do not always capitalize on these learning opportunities. Further, if students lack the necessary expertise to tutor, they will not master the desired knowledge components, even if they take a correct learning path.
* ''Peer plus cognitive tutoring condition (2nd step)'': To increase the probability that students are mastering the correct knowledge components, the cognitive tutor helps peer tutors interact with their partners. Students use both the peer tutor and the cognitive tutor as learning resources.

=== Annotated bibliography ===
* Walker, E., McLaren, B. M., Rummel, N., and Koedinger, K. R. Who Says Three's a Crowd? Using a Cognitive Tutor to Support Peer Tutoring. To appear in the Proceedings of the 13th International Conference on Artificial Intelligence and Education. 2007.
* Walker, E., Rummel, N., McLaren, B. M. & Koedinger, K. R. The Student Becomes the Master: Integrating Peer Tutoring with Cognitive Tutoring. Short paper at the Conference on Computer Supported Collaborative Learning (CSCL-07). Rutgers University, July 16-21, 2007.
* Walker, E., Koedinger, K., McLaren, B. M., & Rummel, N. (2006). Cognitive tutors as research platforms: Extending an established tutoring system for collaborative and metacognitive experimentation. ''Lecture Notes in Computer Science, Volume 4053/2006. Proceedings of the 8th International Conference on Intelligent Tutoring Systems'' (pp. 207-216). Berlin: Springer
* Walker, E. (2005). Mutual peer tutoring: A collaborative addition to the Algebra-1 Cognitive Tutor. Paper presented at the 12th International Conference on Artificial Intelligence and Education (AIED-05, Young Researchers Track), July, 2005, Amsterdam, the Netherlands.

=== References ===

* Aleven, V., McLaren, B., Roll, I., & Koedinger, K. R. (2004). Toward tutoring help seeking: Applying cognitive modelling to meta-cognitive skills. In J. C. Lester, R. M. Vicari & F. Paraguaçu (Eds.), Proceedings of Seventh International Conference on Intelligent Tutoring Systems, ITS 2004 (pp. 227-239). Berlin: Springer.
* Anderson, J. R., Corbett, A. T., Koedinger, K. R., & Pelletier, R. (1995). Cognitive tutors: Lessons learned. Journal of the Learning Sciences, 4(2), 167-207.
* Baker, R. S., Corbett, A. T., & Koedinger, K. R. (2004). Detecting student misuse of intelligent tutoring systems. Paper presented at the Proceedings of the 7th International Conference on Intelligent Tutoring Systems.
* Berg, K. F. (1993). Structured cooperative learning and achievement in a high school mathematics class. Paper presented at the Annual Meeting of the American Educational Research Association, Atlanta, GA.
* Biswas, G., Schwartz, D. L., Leelawong, K., Vye, N., & TAG-V. (2005). Learning by teaching: A new agent paradigm for educational software. Applied Artificial Intelligence, 19, 363–392.
* Carmien, S., Kollar, I., Fischer, G. & Fischer, F. (2006). The interplay of internal and external scripts. In F. Fischer, I. Kollar, H. Mandl &, J. Haake, Scripting computer-supported communication of knowledge. Cognitive, computational, and educational perspectives (pp. 289-311). New York: Springer.
* Fantuzzo, J. W., Riggio, R. E., Connelly, S., & Dimeff, L. A. (1992). Effects of reciprocal peer tutoring on mathematics and school adjustment: A componential analysis. Journal of Educational Psychology, 84(3), 331-339.
* Fuchs, L.S., Fuchs, D., Prentice, K., Burch, M., Hamlett, C.L., Owen, R., & Schroeter, K. (2003). Enhancing third-grade students' mathematical problem solving with self-regulated learning strategies. Journal of Educational Psychology, 95(2), 306-315.
* Harrer, A., McLaren, B. M., Walker, E., Bollen, L., and Sewall, J. (2006). Creating Cognitive Tutors for Collaborative Learning: Steps Toward Realization. User Modeling and User-Adapted Interaction: The Journal of Personalization Research (UMUAI) (2006) 16: 175-209.
* Johnson, D. W. and Johnson, R. T. (1990). Cooperative learning and achievement. In S. Sharan (Ed.), Cooperative learning: Theory and research (pp. 23-37). New York: Praeger.
* King, A., Staffieri, A., & Adelgais, A. (1998). Mutual peer tutoring: Effects of structuring tutorial interaction to scaffold peer learning. Journal of Educational Psychology, 90, 134-152.
* Koedinger, K. R., Anderson, J. R., Hadley, W. H., & Mark, M. A. (1997). Intelligent tutoring goes to school in the big city. International Journal of Artificial Intelligence in Education, 8, 30-43.
* Kollar, I., Fischer, F., & Hesse, F. W. (in press). Collaboration scripts - a conceptual analysis. Educational Psychology Review.
* Ploetzner, R., Dillenbourg, P., Preier, M.,&Traum, D. (1999). Learning by explaining to oneself and to others. In P. Dillenbourg (Ed.), Collaborative learning. Cognitive and computational approaches (pp. 103–121). Amsterdam: Pergamon.
* Ritter, S., Blessing, S. B., & Hadley, W. S. (2002). SBIR Phase I Final Report 2002. Department of Education. Department of Education RFP ED: 84-305S.
* Rummel, N., & Spada, H. (2005). Learning to Collaborate: An Instructional Approach to Promoting Collaborative Problem Solving in Computer-Mediated Settings. Journal of the Learning Sciences, 14(2), 201-241.
* Soller, A.L. (2001). Supporting Social Interaction in an Intelligent Collaborative Learning System. International Journal of Artificial Intelligence in Education, 12, 40-62.
* Teasley, S. D. (1995). The role of talk in children's peer collaborations. Developmental Psychology, 31(2), 207-220.
* Webb, N.M., Troper, J.D., & Fall, R. (1995). Constructive activity and learning in collaborative small groups. Journal of Educational Psychology, 87, 406-423.

=== Connections ===
As this study, [[Rummel Scripted Collaborative Problem Solving]] adds scripted collaborative problem solving to the Cognitive Tutor Algebra. The studies differ in the way collaboration is integrated in the Tutor. First, in the Rummel et al. study, both students first prepare one subtasks of a problem to mutually solve the complex story problem later on. Thus, although the students are experts for different parts of the problem, they have a comparable knowledge level during collaboration. In contrast, in this study, one student prepares to teach his partner. Then, they change roles. Thus, during collaboration, their knowledge level differs. Second, in the Rummel et al. study, collaboration was face to face, whereas this study used a chat tool for interaction.

Similar to the adaptive script component of the Collaborative Problem-Solving Script, the [[The Help Tutor Roll Aleven McLaren|Help Tutor project]] aims at improving students' [[help-seeking behavior]] and at reducing students' tendency to [[game the system]]. Furthermore, both studies contain instructions to teach [[metacognition]]. The metacognitive component in our study instructs students to monitor their interaction in order to improve it in subsequent collaborations; the Help Tutor project asks students to evaluate their need for help in order to improve their help-seeking behavior when learning on the Tutor.

Both in this study and in the [[Reflective Dialogues (Katz)|Reflective Dialogue study]] from Katz, students are asked to engage in reflection following each problem-solving. In this study, the reflection concentrates on the collaborative skills, while in Katz' study, the reflection concentrates on students' domain knowledge of the main principles applied in the problem.

Furthermore, both our study and the [[Help Lite (Aleven, Roll)]] aim at improving conceptual knowledge.

=== Future plans ===
Our future plans for June 2007 - December 2007:
* Run full study
* Analyze process data
* Conference contributions at CSCL and AIED
* Paper publication

[[Category:Study]]

Walker A Peer Tutoring Addition

2007-03-30T22:28:29Z

Erin-Walker: /* Connections */

== Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition ==
''Erin Walker, Bruce M. McLaren, Nikol Rummel, and Ken Koedinger''

=== Summary Tables ===
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''PIs''' || Bruce McLaren, Nikol Rummel
|-
| '''Other Contributers''' ||
* Graduate Student: Erin Walker
* Staff: Jonathan Steinhart
|}
 
'' Pre Study 1''
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''Study Start Date''' || 05/06
|-
| '''Study End Date''' || 06/06
|-
| '''LearnLab Site''' || CWCTC
|-
| '''LearnLab Course''' || Algebra
|-
| '''Number of Students''' || ''N'' = 14
|-
| '''Average # of hours per participant''' || 3 hrs.
|}
 

'' Pre Study 2''
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''Study Start Date''' || 11/06
|-
| '''Study End Date''' || 11/06
|-
| '''LearnLab Site''' || CWCTC
|-
| '''LearnLab Course''' || Algebra
|-
| '''Number of Students''' || ''N'' = 20
|-
| '''Average # of hours per participant''' || 3 hrs.
|}
 

'' Full Study''
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''Study Start Date''' || 04/07
|-
| '''Study End Date''' || 04/07
|-
| '''LearnLab Site''' || CWCTC
|-
| '''LearnLab Course''' || Algebra
|-
| '''Number of Students''' || ''N'' = 70
|-
| '''Average # of hours per participant''' || 3 hrs.
|}
 

=== Abstract ===
In this project, the Algebra I Cognitive Tutor is extended to a [[collaborative learning environment]]: Instead of the computer tutoring a student, students tutor each other, and the [[cognitive tutor]] provides collaborative support. As research has shown, collaborative problem solving and learning has the potential to increase elaboration on the learning content. However, students are not always able to effectively meet the challenges of a collaborative setting. To ensure that students capitalize on collaborative problem solving with the Tutor, a peer tutoring [[collaboration scripts|collaboration script]] was developed that guides their interaction and prompts fruitful [[collaboration]].

In the baseline version of the script (without cognitive tutor support), students in the same class are seated at different computers and take turs tutoring each other. First, they prepare to tutor by solving problems using the regular cognitive tutor. Then, they tutor each other by marking each other right and wrong, monitoring their partner's skills, and give each other hints and feedback. After preparing and tutoring, they reflect on the problems and their interaction, engaging in [[metacognition|metacognitive activities]]. We have expanded the script to include intelligent cognitive support of the peer tutoring, and hope to ultimately expand it to include intelligent collaborative support.

To assess the effectiveness of the script on [[robust learning]], we are conducting a series of [[in vivo experiment]]s in the Algebra LearnLab. In an initial, small scale study (pre study) that served to establish basic effects and to test the procedure in a classroom setting, we compared peer tutoring with reflection elements to peer tutoring without reflection elements. In a second study that served to explore the effects of cognitive tutoring of peer tutoring, we compared peer tutoring plus cognitive tutoring to peer tutoring. In the full study, we will compare these collaborative conditions to an individual condition to assess the effect of the collaborative Tutor extension to regular Tutor use.

=== Background and Significance ===

In our project, we combined two different [[instructional method]]s both of which have been shown to improve students’ learning in mathematics: Learning with intelligent tutoring systems (Koedinger, Anderson, Hadley, & Mark, 1997) and collaborative problem solving (Berg, 1993). The [[Cognitive tutor|Cognitive Tutor]] Algebra that was used in our study is a tutor for mathematics instruction at the high school level. Students learn with the Tutor during part of the regular classroom sessions. The Tutor's main features are immediate error feedback, the possibility to ask for a hint when encountering impasses, and knowledge tracing, i.e. the Tutor creates and updates a model of the student’s knowledge and selects new problems tailored to the student’s knowledge level. Although several studies have proven its effectiveness, students do not always benefit from learning with the Tutor. First, because the Tutor places emphasis on learning [[procedural|procedural problem solving skills]], yet a deep understanding of underlying mathematical [[conceptual knowledge|concepts]] is not necessarily achieved (Anderson, Corbett, Koedinger, & Pelletier, 1995). Second, students do not always make good use of the learning opportunities provided by the Cognitive Tutor (e.g. [[help abuse]], see Aleven, McLaren, Roll, & Koedinger, 2004; [[game the system|gaming the system]], Baker, Corbett, & Koedinger, 2004). So far, the Cognitive Tutor has been used in an individual learning setting only. However, as research on collaborative learning has shown, collaboration can yield elaboration of learning content (Teasley, 1995), thus this could be a promising approach to reduce the Tutor’s shortcomings. On the other hand, students are not always able to effectively meet the challenges of a collaborative setting (Rummel & Spada, 2005). [[Collaboration scripts]] have proven effective in helping people meet the challenges encountered when learning or working collaboratively (Kollar, Fischer, & Hesse, in press).

Combining collaborative activities with intelligent tutoring might combine the benefits of both approaches. Collaborative interaction could augment the effects of an existing ITS by adding deeper interaction, and intelligent tutoring support could improve the quality of student collaboration by providing guidance to students as they attempt to follow a collaboration script. One way to incorporate collaboration into an ITS is to place the intelligent tutoring system in the role of one of the collaborators and have a student interact with this agent. For example, Biswas, Schwartz, Leelawong, Vye, and the TAG-V (2005) have the student collaborate with an agent (“Betty”) in a peer tutoring scenario. The student teaches Betty about ecosystems with the help of a second mentoring agent (“Mr. Davis”), and it has been shown that a student who takes the role of a teacher in this system benefits more than a student who takes the role of a learner in a parallel traditional tutoring system. However, this system does not yet support natural language interaction between the agents involved, and therefore does not take full advantage of important collaborative interaction mechanisms that can increase learning. Another way to combine collaboration and tutoring is to use the ITS to structure and tutor the student collaboration, by connecting collaborative tools with ITS components (see Harrer, McLaren, Walker, Bollen, & Sewall, 2006), and developing a model of good and ineffective collaboration in order to provide on-line hints and feedback (see Soller, 2001). Most research into computer tutoring of the collaboration between two students is still in a preliminary stage.

We integrate collaborative learning with an ITS using a peer tutoring framework. Similar to Biswas et al. (2005), we put the student in the role of the tutor, and intend to have the cognitive elements of the tutoring supported by another agent. However, unlike Biswas et al., (2005), the teachable agent is a real student. Therefore, we have intelligent tutoring components monitoring and providing feedback on student collaboration in addition to student cognition, like Harrer et al. (2006). Our ultimate goal is to allow students to tutor each other through the interface of an ITS, supported by both cognitive and collaborative tutoring.

=== Glossary ===
See [[:Category:Peer Tutoring|Peer Tutoring Glossary]]

=== Research question ===
Does [[collaboration]] – and in particular [[collaboration scripts|scripted collaboration]] – improve students’ [[robust learning]] in the domain of algebra?

Do [[instructional method|instructional activities]] that involve three agents (a student, a peer tutor, and a computer tutor) increase [[robust learning]] compared to instructional activities that involve two agents (student and peer tutor, student and computer tutor)?

=== Independent variables ===
We vary the [[agent]]s involved in the interaction using the [[instructional method|mode of instruction]]. There are four conditions:
* Individual condition: Students solved problems with the Algebra I Cognitive Tutor in the regular fashion (i.e. computer program as additional [[agent]]).
* Peer tutoring condition: Students tutored each other using the cognitive tutor interface (i.e., another peer as additional [[agent]]). They went through two phases:
** a preparation phase, where students prepared to tutor while solving problems with the Algebra I Cognitive Tutor in the regular fashion
** a collaboration phase, where students take turns tutoring each other. Peer tutors can mark answers right or wrong, monitor their partner's skills, and give hints and feedback
* Peer tutoring plus reflection: As in the peer tutoring condition, students go through a preparation phase and then tutor each other in the collaboration phase. However, students also go through a series of [[metacognition|metacognitive]] self-reflection activities to encourage them to engage in [[elaborative interaction]]
* Peer tutoring plus cognitive tutoring: Students participate in the peer tutoring plus reflection condition, but during the collaboration phase, peer tutors receive cognitive tutor hints and feedback (i.e., a peer and computer program as additional [[agent]]s). This feedback contains two components:
** a collaborative prompt that encourages students to interact with each other
** a cognitive hint targeted at the particular mistake the peer tutee is making

=== Hypothesis ===
1. We expect the collaborative conditions to outperform the individual conditions in measures of robust learning of the algebra content.

2. We epect the peer tutoring plus cognitive tutoring condition to outperform the peer tutoring alone conditions in measures of robust learning of the algebra content

=== Dependent variables ===
* ''Near [[transfer]], immediate'': During training, student progress on training problems are be analyzed.
* ''Near [[transfer]], [[long-term retention|retention]]'': Students a are given a posttest immediately after the study on isomorphic problems
* ''Far [[transfer]]'': This paper and pencil test assessed students' understanding of the main mathematical concepts from the learning phase. The [[conceptual tasks]] students had to solve tapped the same knowledge components as the problems in instruction; however, the problems where non-isomorphic to those in the instruction, thus demanded students to flexibly apply their knowledge to problems with a new format.
* ''Acceleration of future learning'': Student learning on future equation solving units will be measured.

To compare [[collaboration skill]]s of students, we will be conducting an analysis of student dialogues durng the learning phase.

=== Explanation ===
This study is part of the [[Interactive_Communication|Interactive Communication cluster]], and its hypothesis is a specialization of the IC cluster’s central hypothesis. We assume that the instruction is in or above the student’s ZPD. Here is what we expect to happen in the three conditions:

* In the peer tutoring condition, the peer tutor’s help should increase learning compared to a student learning with the cognitive tutor. However, the student may make too many errors and/or require too much communication with the second agent. The peer tutor may not be able to provide the student with the help he/she needs. Learning will not be optimized.
* In the cognitive+peer tutoring condition, the peer tutor’s help should provide the additional benefits of interactive communication, and the cognitive tutor’s help should ensure that both the student and the peer tutor get the problem solving assistance they need. The support of the cognitive tutor and interaction with the student will also increase the peer tutor’s learning.
The target path will have the students learning-by-doing and learning-by-teaching, with the help of the computer agent so students do not make too many errors or require too much communication.

=== Findings ===
'''Pre-study 1:'''

Our first study compared the peer tutoring to peer tutoring plus reflection conditions. Using pretest and posttest scores, we conducted a two-way (condition x test-time) repeated-measure ANOVA, with test-time as the repeated measure. Posttest scores were significantly higher than pretest scores in both the tutoring and the tutoring+reflection condition (F (1,12) = 15.25, p < .002, η² = 0.56), but there were no significant differences between conditions, and no interaction (see Table 2). To further examine what occurred during the collaboration phase we turned to log data and notes from classroom observation. During peer tutoring, students appeared engaged, and did exhibit many of the positive collaborative behaviors that we were attempting to encourage with our script and that have been shown to correlate with knowledge construction and self-reflection. However, we observed that peer tutors struggled to provide tutees with answers, and did not connect the preparation that they had done with the collaboration phase. For instance, they often did not consult their answer printouts when they did not know the next problem step and thus had to rely on teacher assistance to solve a problem. As a result, tutees skipped problems without completing them correctly. This undesirable behavior differed between the two conditions. Students in the tutoring condition attempted more problems than students in the tutoring+reflection condition, and appeared to complete more problems as well. The average number of problems completed by dyads in the tutoring+reflection condition was low; students in this group took an average of 11 minutes to complete a single problem, compared to a 6 minute average in the tutoring condition. Students in the tutoring condition tended to skip problems they could not solve, completing less than 60% of the problems they attempted. Immediately before skipping a problem, students would generally state their inability to solve it, “I don’t know how to do this one,“ or their lack of motivation, “Just do something and I’ll agree or something.” If students skip problems, they may not learn how to solve difficult problems. However, if they do not complete many problems, they may not be sufficiently exposed to all the skills involved in the unit, and will be given fewer opportunities to master them.

'''Results for Pre-study 2 and Study 1 coming soon.'''

=== Explanation ===
This study is part of the [[Interactive_Communication|Interactive Communication cluster]], and its hypothesis is a specialization of the IC cluster’s central hypothesis. According to the IC cluster’s hypothesis, instruction that yields [[robust learning]] should be designed to have the right target paths. Furthermore, it should improve the students’ path choice in the [[learning event space]], i.e., increasing the probability that students take correct paths and decreasing the probability that students take alternative paths.

By enhancing the Cognitive Tutor with collaboration, the study tried to reach these two goals in two steps:

* ''Individual condition'': The existing Cognitive Tutor already offers correct learning paths. However, individual learning on the Cognitive Tutor has several shortcomings. For example, students don’t have the opportunity to reflect on the underlying mathematical [[conceptual knowledge|concepts]] in natural language interaction in order to gain a deeper understanding. In addition, students do not always use the offered learning paths (as the opportunity to reflect on on-demand hints and error feedback) effectively, thus often do not engage in [[sense making]] processes
* ''Peer tutoring condition (1st step)'': By enhancing the Algebra I Cognitive Tutor to be a collaborative learning environment, we added a learning resource – the learning partner. This adds further correct learning paths to the [[learning event space]], for instance, learning by giving explanations, the possibility to request help, and learning by knowledge co-construction. However, similarly to the learning paths in an individual setting, students do not always capitalize on these learning opportunities. Further, if students lack the necessary expertise to tutor, they will not master the desired knowledge components, even if they take a correct learning path.
* ''Peer plus cognitive tutoring condition (2nd step)'': To increase the probability that students are mastering the correct knowledge components, the cognitive tutor helps peer tutors interact with their partners. Students use both the peer tutor and the cognitive tutor as learning resources.

=== Annotated bibliography ===
* Walker, E., McLaren, B. M., Rummel, N., and Koedinger, K. R. Who Says Three's a Crowd? Using a Cognitive Tutor to Support Peer Tutoring. To appear in the Proceedings of the 13th International Conference on Artificial Intelligence and Education. 2007.
* Walker, E., Rummel, N., McLaren, B. M. & Koedinger, K. R. The Student Becomes the Master: Integrating Peer Tutoring with Cognitive Tutoring. Short paper at the Conference on Computer Supported Collaborative Learning (CSCL-07). Rutgers University, July 16-21, 2007.
* Walker, E., Koedinger, K., McLaren, B. M., & Rummel, N. (2006). Cognitive tutors as research platforms: Extending an established tutoring system for collaborative and metacognitive experimentation. ''Lecture Notes in Computer Science, Volume 4053/2006. Proceedings of the 8th International Conference on Intelligent Tutoring Systems'' (pp. 207-216). Berlin: Springer
* Walker, E. (2005). Mutual peer tutoring: A collaborative addition to the Algebra-1 Cognitive Tutor. Paper presented at the 12th International Conference on Artificial Intelligence and Education (AIED-05, Young Researchers Track), July, 2005, Amsterdam, the Netherlands.

=== References ===

* Aleven, V., McLaren, B., Roll, I., & Koedinger, K. R. (2004). Toward tutoring help seeking: Applying cognitive modelling to meta-cognitive skills. In J. C. Lester, R. M. Vicari & F. Paraguaçu (Eds.), Proceedings of Seventh International Conference on Intelligent Tutoring Systems, ITS 2004 (pp. 227-239). Berlin: Springer.
* Anderson, J. R., Corbett, A. T., Koedinger, K. R., & Pelletier, R. (1995). Cognitive tutors: Lessons learned. Journal of the Learning Sciences, 4(2), 167-207.
* Baker, R. S., Corbett, A. T., & Koedinger, K. R. (2004). Detecting student misuse of intelligent tutoring systems. Paper presented at the Proceedings of the 7th International Conference on Intelligent Tutoring Systems.
* Berg, K. F. (1993). Structured cooperative learning and achievement in a high school mathematics class. Paper presented at the Annual Meeting of the American Educational Research Association, Atlanta, GA.
* Biswas, G., Schwartz, D. L., Leelawong, K., Vye, N., & TAG-V. (2005). Learning by teaching: A new agent paradigm for educational software. Applied Artificial Intelligence, 19, 363–392.
* Carmien, S., Kollar, I., Fischer, G. & Fischer, F. (2006). The interplay of internal and external scripts. In F. Fischer, I. Kollar, H. Mandl &, J. Haake, Scripting computer-supported communication of knowledge. Cognitive, computational, and educational perspectives (pp. 289-311). New York: Springer.
* Fantuzzo, J. W., Riggio, R. E., Connelly, S., & Dimeff, L. A. (1992). Effects of reciprocal peer tutoring on mathematics and school adjustment: A componential analysis. Journal of Educational Psychology, 84(3), 331-339.
* Fuchs, L.S., Fuchs, D., Prentice, K., Burch, M., Hamlett, C.L., Owen, R., & Schroeter, K. (2003). Enhancing third-grade students' mathematical problem solving with self-regulated learning strategies. Journal of Educational Psychology, 95(2), 306-315.
* Harrer, A., McLaren, B. M., Walker, E., Bollen, L., and Sewall, J. (2006). Creating Cognitive Tutors for Collaborative Learning: Steps Toward Realization. User Modeling and User-Adapted Interaction: The Journal of Personalization Research (UMUAI) (2006) 16: 175-209.
* Johnson, D. W. and Johnson, R. T. (1990). Cooperative learning and achievement. In S. Sharan (Ed.), Cooperative learning: Theory and research (pp. 23-37). New York: Praeger.
* King, A., Staffieri, A., & Adelgais, A. (1998). Mutual peer tutoring: Effects of structuring tutorial interaction to scaffold peer learning. Journal of Educational Psychology, 90, 134-152.
* Koedinger, K. R., Anderson, J. R., Hadley, W. H., & Mark, M. A. (1997). Intelligent tutoring goes to school in the big city. International Journal of Artificial Intelligence in Education, 8, 30-43.
* Kollar, I., Fischer, F., & Hesse, F. W. (in press). Collaboration scripts - a conceptual analysis. Educational Psychology Review.
* Ploetzner, R., Dillenbourg, P., Preier, M.,&Traum, D. (1999). Learning by explaining to oneself and to others. In P. Dillenbourg (Ed.), Collaborative learning. Cognitive and computational approaches (pp. 103–121). Amsterdam: Pergamon.
* Ritter, S., Blessing, S. B., & Hadley, W. S. (2002). SBIR Phase I Final Report 2002. Department of Education. Department of Education RFP ED: 84-305S.
* Rummel, N., & Spada, H. (2005). Learning to Collaborate: An Instructional Approach to Promoting Collaborative Problem Solving in Computer-Mediated Settings. Journal of the Learning Sciences, 14(2), 201-241.
* Soller, A.L. (2001). Supporting Social Interaction in an Intelligent Collaborative Learning System. International Journal of Artificial Intelligence in Education, 12, 40-62.
* Teasley, S. D. (1995). The role of talk in children's peer collaborations. Developmental Psychology, 31(2), 207-220.
* Webb, N.M., Troper, J.D., & Fall, R. (1995). Constructive activity and learning in collaborative small groups. Journal of Educational Psychology, 87, 406-423.

=== Connections ===
As this study, [[Rummel Scripted Collaborative Problem Solving]] adds scripted collaborative problem solving to the Cognitive Tutor Algebra. The studies differ in the way collaboration is integrated in the Tutor. First, in the Rummel et al. study, both students first prepare one subtasks of a problem to mutually solve the complex story problem later on. Thus, although the students are experts for different parts of the problem, they have a comparable knowledge level during collaboration. In contrast, in this study, one student prepares to teach his partner. Then, they change roles. Thus, during collaboration, their knowledge level differs. Second, in the Rummel et al. study, collaboration was face to face, whereas this study used a chat tool for interaction.

Similar to the adaptive script component of the Collaborative Problem-Solving Script, the [[The Help Tutor Roll Aleven McLaren|Help Tutor project]] aims at improving students' [[help-seeking behavior]] and at reducing students' tendency to [[game the system]]. Furthermore, both studies contain instructions to teach [[metacognition]]. The metacognitive component in our study instructs students to monitor their interaction in order to improve it in subsequent collaborations; the Help Tutor project asks students to evaluate their need for help in order to improve their help-seeking behavior when learning on the Tutor.

Both in this study and in the [[Reflective Dialogues (Katz)|Reflective Dialogue study]] from Katz, students are asked to engage in reflection following each problem-solving. In this study, the reflection concentrates on the collaborative skills, while in Katz' study, the reflection concentrates on students' domain knowledge of the main principles applied in the problem.

Furthermore, both our study and the [[Help Lite (Aleven, Roll)]] aim at improving conceptual knowledge.

==== Future plans ====
Our future plans for June 2007 - December 2007:
* Run full study
* Analyze process data
* Conference contributions at CSCL and AIED
* Paper publication

[[Category:Study]]

Walker A Peer Tutoring Addition

2007-03-30T22:27:33Z

Erin-Walker: /* References */

== Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition ==
''Erin Walker, Bruce M. McLaren, Nikol Rummel, and Ken Koedinger''

=== Summary Tables ===
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''PIs''' || Bruce McLaren, Nikol Rummel
|-
| '''Other Contributers''' ||
* Graduate Student: Erin Walker
* Staff: Jonathan Steinhart
|}
 
'' Pre Study 1''
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''Study Start Date''' || 05/06
|-
| '''Study End Date''' || 06/06
|-
| '''LearnLab Site''' || CWCTC
|-
| '''LearnLab Course''' || Algebra
|-
| '''Number of Students''' || ''N'' = 14
|-
| '''Average # of hours per participant''' || 3 hrs.
|}
 

'' Pre Study 2''
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''Study Start Date''' || 11/06
|-
| '''Study End Date''' || 11/06
|-
| '''LearnLab Site''' || CWCTC
|-
| '''LearnLab Course''' || Algebra
|-
| '''Number of Students''' || ''N'' = 20
|-
| '''Average # of hours per participant''' || 3 hrs.
|}
 

'' Full Study''
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''Study Start Date''' || 04/07
|-
| '''Study End Date''' || 04/07
|-
| '''LearnLab Site''' || CWCTC
|-
| '''LearnLab Course''' || Algebra
|-
| '''Number of Students''' || ''N'' = 70
|-
| '''Average # of hours per participant''' || 3 hrs.
|}
 

=== Abstract ===
In this project, the Algebra I Cognitive Tutor is extended to a [[collaborative learning environment]]: Instead of the computer tutoring a student, students tutor each other, and the [[cognitive tutor]] provides collaborative support. As research has shown, collaborative problem solving and learning has the potential to increase elaboration on the learning content. However, students are not always able to effectively meet the challenges of a collaborative setting. To ensure that students capitalize on collaborative problem solving with the Tutor, a peer tutoring [[collaboration scripts|collaboration script]] was developed that guides their interaction and prompts fruitful [[collaboration]].

In the baseline version of the script (without cognitive tutor support), students in the same class are seated at different computers and take turs tutoring each other. First, they prepare to tutor by solving problems using the regular cognitive tutor. Then, they tutor each other by marking each other right and wrong, monitoring their partner's skills, and give each other hints and feedback. After preparing and tutoring, they reflect on the problems and their interaction, engaging in [[metacognition|metacognitive activities]]. We have expanded the script to include intelligent cognitive support of the peer tutoring, and hope to ultimately expand it to include intelligent collaborative support.

To assess the effectiveness of the script on [[robust learning]], we are conducting a series of [[in vivo experiment]]s in the Algebra LearnLab. In an initial, small scale study (pre study) that served to establish basic effects and to test the procedure in a classroom setting, we compared peer tutoring with reflection elements to peer tutoring without reflection elements. In a second study that served to explore the effects of cognitive tutoring of peer tutoring, we compared peer tutoring plus cognitive tutoring to peer tutoring. In the full study, we will compare these collaborative conditions to an individual condition to assess the effect of the collaborative Tutor extension to regular Tutor use.

=== Background and Significance ===

In our project, we combined two different [[instructional method]]s both of which have been shown to improve students’ learning in mathematics: Learning with intelligent tutoring systems (Koedinger, Anderson, Hadley, & Mark, 1997) and collaborative problem solving (Berg, 1993). The [[Cognitive tutor|Cognitive Tutor]] Algebra that was used in our study is a tutor for mathematics instruction at the high school level. Students learn with the Tutor during part of the regular classroom sessions. The Tutor's main features are immediate error feedback, the possibility to ask for a hint when encountering impasses, and knowledge tracing, i.e. the Tutor creates and updates a model of the student’s knowledge and selects new problems tailored to the student’s knowledge level. Although several studies have proven its effectiveness, students do not always benefit from learning with the Tutor. First, because the Tutor places emphasis on learning [[procedural|procedural problem solving skills]], yet a deep understanding of underlying mathematical [[conceptual knowledge|concepts]] is not necessarily achieved (Anderson, Corbett, Koedinger, & Pelletier, 1995). Second, students do not always make good use of the learning opportunities provided by the Cognitive Tutor (e.g. [[help abuse]], see Aleven, McLaren, Roll, & Koedinger, 2004; [[game the system|gaming the system]], Baker, Corbett, & Koedinger, 2004). So far, the Cognitive Tutor has been used in an individual learning setting only. However, as research on collaborative learning has shown, collaboration can yield elaboration of learning content (Teasley, 1995), thus this could be a promising approach to reduce the Tutor’s shortcomings. On the other hand, students are not always able to effectively meet the challenges of a collaborative setting (Rummel & Spada, 2005). [[Collaboration scripts]] have proven effective in helping people meet the challenges encountered when learning or working collaboratively (Kollar, Fischer, & Hesse, in press).

Combining collaborative activities with intelligent tutoring might combine the benefits of both approaches. Collaborative interaction could augment the effects of an existing ITS by adding deeper interaction, and intelligent tutoring support could improve the quality of student collaboration by providing guidance to students as they attempt to follow a collaboration script. One way to incorporate collaboration into an ITS is to place the intelligent tutoring system in the role of one of the collaborators and have a student interact with this agent. For example, Biswas, Schwartz, Leelawong, Vye, and the TAG-V (2005) have the student collaborate with an agent (“Betty”) in a peer tutoring scenario. The student teaches Betty about ecosystems with the help of a second mentoring agent (“Mr. Davis”), and it has been shown that a student who takes the role of a teacher in this system benefits more than a student who takes the role of a learner in a parallel traditional tutoring system. However, this system does not yet support natural language interaction between the agents involved, and therefore does not take full advantage of important collaborative interaction mechanisms that can increase learning. Another way to combine collaboration and tutoring is to use the ITS to structure and tutor the student collaboration, by connecting collaborative tools with ITS components (see Harrer, McLaren, Walker, Bollen, & Sewall, 2006), and developing a model of good and ineffective collaboration in order to provide on-line hints and feedback (see Soller, 2001). Most research into computer tutoring of the collaboration between two students is still in a preliminary stage.

We integrate collaborative learning with an ITS using a peer tutoring framework. Similar to Biswas et al. (2005), we put the student in the role of the tutor, and intend to have the cognitive elements of the tutoring supported by another agent. However, unlike Biswas et al., (2005), the teachable agent is a real student. Therefore, we have intelligent tutoring components monitoring and providing feedback on student collaboration in addition to student cognition, like Harrer et al. (2006). Our ultimate goal is to allow students to tutor each other through the interface of an ITS, supported by both cognitive and collaborative tutoring.

=== Glossary ===
See [[:Category:Peer Tutoring|Peer Tutoring Glossary]]

=== Research question ===
Does [[collaboration]] – and in particular [[collaboration scripts|scripted collaboration]] – improve students’ [[robust learning]] in the domain of algebra?

Do [[instructional method|instructional activities]] that involve three agents (a student, a peer tutor, and a computer tutor) increase [[robust learning]] compared to instructional activities that involve two agents (student and peer tutor, student and computer tutor)?

=== Independent variables ===
We vary the [[agent]]s involved in the interaction using the [[instructional method|mode of instruction]]. There are four conditions:
* Individual condition: Students solved problems with the Algebra I Cognitive Tutor in the regular fashion (i.e. computer program as additional [[agent]]).
* Peer tutoring condition: Students tutored each other using the cognitive tutor interface (i.e., another peer as additional [[agent]]). They went through two phases:
** a preparation phase, where students prepared to tutor while solving problems with the Algebra I Cognitive Tutor in the regular fashion
** a collaboration phase, where students take turns tutoring each other. Peer tutors can mark answers right or wrong, monitor their partner's skills, and give hints and feedback
* Peer tutoring plus reflection: As in the peer tutoring condition, students go through a preparation phase and then tutor each other in the collaboration phase. However, students also go through a series of [[metacognition|metacognitive]] self-reflection activities to encourage them to engage in [[elaborative interaction]]
* Peer tutoring plus cognitive tutoring: Students participate in the peer tutoring plus reflection condition, but during the collaboration phase, peer tutors receive cognitive tutor hints and feedback (i.e., a peer and computer program as additional [[agent]]s). This feedback contains two components:
** a collaborative prompt that encourages students to interact with each other
** a cognitive hint targeted at the particular mistake the peer tutee is making

=== Hypothesis ===
1. We expect the collaborative conditions to outperform the individual conditions in measures of robust learning of the algebra content.

2. We epect the peer tutoring plus cognitive tutoring condition to outperform the peer tutoring alone conditions in measures of robust learning of the algebra content

=== Dependent variables ===
* ''Near [[transfer]], immediate'': During training, student progress on training problems are be analyzed.
* ''Near [[transfer]], [[long-term retention|retention]]'': Students a are given a posttest immediately after the study on isomorphic problems
* ''Far [[transfer]]'': This paper and pencil test assessed students' understanding of the main mathematical concepts from the learning phase. The [[conceptual tasks]] students had to solve tapped the same knowledge components as the problems in instruction; however, the problems where non-isomorphic to those in the instruction, thus demanded students to flexibly apply their knowledge to problems with a new format.
* ''Acceleration of future learning'': Student learning on future equation solving units will be measured.

To compare [[collaboration skill]]s of students, we will be conducting an analysis of student dialogues durng the learning phase.

=== Explanation ===
This study is part of the [[Interactive_Communication|Interactive Communication cluster]], and its hypothesis is a specialization of the IC cluster’s central hypothesis. We assume that the instruction is in or above the student’s ZPD. Here is what we expect to happen in the three conditions:

* In the peer tutoring condition, the peer tutor’s help should increase learning compared to a student learning with the cognitive tutor. However, the student may make too many errors and/or require too much communication with the second agent. The peer tutor may not be able to provide the student with the help he/she needs. Learning will not be optimized.
* In the cognitive+peer tutoring condition, the peer tutor’s help should provide the additional benefits of interactive communication, and the cognitive tutor’s help should ensure that both the student and the peer tutor get the problem solving assistance they need. The support of the cognitive tutor and interaction with the student will also increase the peer tutor’s learning.
The target path will have the students learning-by-doing and learning-by-teaching, with the help of the computer agent so students do not make too many errors or require too much communication.

=== Findings ===
'''Pre-study 1:'''

Our first study compared the peer tutoring to peer tutoring plus reflection conditions. Using pretest and posttest scores, we conducted a two-way (condition x test-time) repeated-measure ANOVA, with test-time as the repeated measure. Posttest scores were significantly higher than pretest scores in both the tutoring and the tutoring+reflection condition (F (1,12) = 15.25, p < .002, η² = 0.56), but there were no significant differences between conditions, and no interaction (see Table 2). To further examine what occurred during the collaboration phase we turned to log data and notes from classroom observation. During peer tutoring, students appeared engaged, and did exhibit many of the positive collaborative behaviors that we were attempting to encourage with our script and that have been shown to correlate with knowledge construction and self-reflection. However, we observed that peer tutors struggled to provide tutees with answers, and did not connect the preparation that they had done with the collaboration phase. For instance, they often did not consult their answer printouts when they did not know the next problem step and thus had to rely on teacher assistance to solve a problem. As a result, tutees skipped problems without completing them correctly. This undesirable behavior differed between the two conditions. Students in the tutoring condition attempted more problems than students in the tutoring+reflection condition, and appeared to complete more problems as well. The average number of problems completed by dyads in the tutoring+reflection condition was low; students in this group took an average of 11 minutes to complete a single problem, compared to a 6 minute average in the tutoring condition. Students in the tutoring condition tended to skip problems they could not solve, completing less than 60% of the problems they attempted. Immediately before skipping a problem, students would generally state their inability to solve it, “I don’t know how to do this one,“ or their lack of motivation, “Just do something and I’ll agree or something.” If students skip problems, they may not learn how to solve difficult problems. However, if they do not complete many problems, they may not be sufficiently exposed to all the skills involved in the unit, and will be given fewer opportunities to master them.

'''Results for Pre-study 2 and Study 1 coming soon.'''

=== Explanation ===
This study is part of the [[Interactive_Communication|Interactive Communication cluster]], and its hypothesis is a specialization of the IC cluster’s central hypothesis. According to the IC cluster’s hypothesis, instruction that yields [[robust learning]] should be designed to have the right target paths. Furthermore, it should improve the students’ path choice in the [[learning event space]], i.e., increasing the probability that students take correct paths and decreasing the probability that students take alternative paths.

By enhancing the Cognitive Tutor with collaboration, the study tried to reach these two goals in two steps:

* ''Individual condition'': The existing Cognitive Tutor already offers correct learning paths. However, individual learning on the Cognitive Tutor has several shortcomings. For example, students don’t have the opportunity to reflect on the underlying mathematical [[conceptual knowledge|concepts]] in natural language interaction in order to gain a deeper understanding. In addition, students do not always use the offered learning paths (as the opportunity to reflect on on-demand hints and error feedback) effectively, thus often do not engage in [[sense making]] processes
* ''Peer tutoring condition (1st step)'': By enhancing the Algebra I Cognitive Tutor to be a collaborative learning environment, we added a learning resource – the learning partner. This adds further correct learning paths to the [[learning event space]], for instance, learning by giving explanations, the possibility to request help, and learning by knowledge co-construction. However, similarly to the learning paths in an individual setting, students do not always capitalize on these learning opportunities. Further, if students lack the necessary expertise to tutor, they will not master the desired knowledge components, even if they take a correct learning path.
* ''Peer plus cognitive tutoring condition (2nd step)'': To increase the probability that students are mastering the correct knowledge components, the cognitive tutor helps peer tutors interact with their partners. Students use both the peer tutor and the cognitive tutor as learning resources.

=== Annotated bibliography ===
* Walker, E., McLaren, B. M., Rummel, N., and Koedinger, K. R. Who Says Three's a Crowd? Using a Cognitive Tutor to Support Peer Tutoring. To appear in the Proceedings of the 13th International Conference on Artificial Intelligence and Education. 2007.
* Walker, E., Rummel, N., McLaren, B. M. & Koedinger, K. R. The Student Becomes the Master: Integrating Peer Tutoring with Cognitive Tutoring. Short paper at the Conference on Computer Supported Collaborative Learning (CSCL-07). Rutgers University, July 16-21, 2007.
* Walker, E., Koedinger, K., McLaren, B. M., & Rummel, N. (2006). Cognitive tutors as research platforms: Extending an established tutoring system for collaborative and metacognitive experimentation. ''Lecture Notes in Computer Science, Volume 4053/2006. Proceedings of the 8th International Conference on Intelligent Tutoring Systems'' (pp. 207-216). Berlin: Springer
* Walker, E. (2005). Mutual peer tutoring: A collaborative addition to the Algebra-1 Cognitive Tutor. Paper presented at the 12th International Conference on Artificial Intelligence and Education (AIED-05, Young Researchers Track), July, 2005, Amsterdam, the Netherlands.

=== References ===

* Aleven, V., McLaren, B., Roll, I., & Koedinger, K. R. (2004). Toward tutoring help seeking: Applying cognitive modelling to meta-cognitive skills. In J. C. Lester, R. M. Vicari & F. Paraguaçu (Eds.), Proceedings of Seventh International Conference on Intelligent Tutoring Systems, ITS 2004 (pp. 227-239). Berlin: Springer.
* Anderson, J. R., Corbett, A. T., Koedinger, K. R., & Pelletier, R. (1995). Cognitive tutors: Lessons learned. Journal of the Learning Sciences, 4(2), 167-207.
* Baker, R. S., Corbett, A. T., & Koedinger, K. R. (2004). Detecting student misuse of intelligent tutoring systems. Paper presented at the Proceedings of the 7th International Conference on Intelligent Tutoring Systems.
* Berg, K. F. (1993). Structured cooperative learning and achievement in a high school mathematics class. Paper presented at the Annual Meeting of the American Educational Research Association, Atlanta, GA.
* Biswas, G., Schwartz, D. L., Leelawong, K., Vye, N., & TAG-V. (2005). Learning by teaching: A new agent paradigm for educational software. Applied Artificial Intelligence, 19, 363–392.
* Carmien, S., Kollar, I., Fischer, G. & Fischer, F. (2006). The interplay of internal and external scripts. In F. Fischer, I. Kollar, H. Mandl &, J. Haake, Scripting computer-supported communication of knowledge. Cognitive, computational, and educational perspectives (pp. 289-311). New York: Springer.
* Fantuzzo, J. W., Riggio, R. E., Connelly, S., & Dimeff, L. A. (1992). Effects of reciprocal peer tutoring on mathematics and school adjustment: A componential analysis. Journal of Educational Psychology, 84(3), 331-339.
* Fuchs, L.S., Fuchs, D., Prentice, K., Burch, M., Hamlett, C.L., Owen, R., & Schroeter, K. (2003). Enhancing third-grade students' mathematical problem solving with self-regulated learning strategies. Journal of Educational Psychology, 95(2), 306-315.
* Harrer, A., McLaren, B. M., Walker, E., Bollen, L., and Sewall, J. (2006). Creating Cognitive Tutors for Collaborative Learning: Steps Toward Realization. User Modeling and User-Adapted Interaction: The Journal of Personalization Research (UMUAI) (2006) 16: 175-209.
* Johnson, D. W. and Johnson, R. T. (1990). Cooperative learning and achievement. In S. Sharan (Ed.), Cooperative learning: Theory and research (pp. 23-37). New York: Praeger.
* King, A., Staffieri, A., & Adelgais, A. (1998). Mutual peer tutoring: Effects of structuring tutorial interaction to scaffold peer learning. Journal of Educational Psychology, 90, 134-152.
* Koedinger, K. R., Anderson, J. R., Hadley, W. H., & Mark, M. A. (1997). Intelligent tutoring goes to school in the big city. International Journal of Artificial Intelligence in Education, 8, 30-43.
* Kollar, I., Fischer, F., & Hesse, F. W. (in press). Collaboration scripts - a conceptual analysis. Educational Psychology Review.
* Ploetzner, R., Dillenbourg, P., Preier, M.,&Traum, D. (1999). Learning by explaining to oneself and to others. In P. Dillenbourg (Ed.), Collaborative learning. Cognitive and computational approaches (pp. 103–121). Amsterdam: Pergamon.
* Ritter, S., Blessing, S. B., & Hadley, W. S. (2002). SBIR Phase I Final Report 2002. Department of Education. Department of Education RFP ED: 84-305S.
* Rummel, N., & Spada, H. (2005). Learning to Collaborate: An Instructional Approach to Promoting Collaborative Problem Solving in Computer-Mediated Settings. Journal of the Learning Sciences, 14(2), 201-241.
* Soller, A.L. (2001). Supporting Social Interaction in an Intelligent Collaborative Learning System. International Journal of Artificial Intelligence in Education, 12, 40-62.
* Teasley, S. D. (1995). The role of talk in children's peer collaborations. Developmental Psychology, 31(2), 207-220.
* Webb, N.M., Troper, J.D., & Fall, R. (1995). Constructive activity and learning in collaborative small groups. Journal of Educational Psychology, 87, 406-423.

=== Connections ===
As this study, [[Rummel Scripted Collaborative Problem Solving]] adds scripted collaborative problem solving to the Cognitive Tutor Algebra. The studies differ in the way collaboration is integrated in the Tutor. First, in the Rummel et al. study, both students first prepare one subtasks of a problem to mutually solve the complex story problem later on. Thus, although the students are experts for different parts of the problem, they have a comparable knowledge level during collaboration. In contrast, in this study, one student prepares to teach his partner. Then, they change roles. Thus, during collaboration, their knowledge level differs. Second, in the Rummel et al. study, collaboration was face to face, whereas this study used a chat tool for interaction.

Similar to the adaptive script component of the Collaborative Problem-Solving Script, the [[The Help Tutor Roll Aleven McLaren|Help Tutor project]] aims at improving students' [[help-seeking behavior]] and at reducing students' tendency to [[game the system]]. Furthermore, both studies contain instructions to teach [[metacognition]]. The metacognitive component in our study instructs students to monitor their interaction in order to improve it in subsequent collaborations; the Help Tutor project asks students to evaluate their need for help in order to improve their help-seeking behavior when learning on the Tutor.

Both in this study and in the [[Reflective Dialogues (Katz)|Reflective Dialogue study]] from Katz, students are asked to engage in reflection following each problem-solving. In this study, the reflection concentrates on the collaborative skills, while in Katz' study, the reflection concentrates on students' domain knowledge of the main principles applied in the problem.

Furthermore, both our study and the [[Help Lite (Aleven, Roll)]] aim at improving conceptual knowledge.

Walker A Peer Tutoring Addition

2007-03-30T22:25:23Z

Erin-Walker: /* Findings */

== Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition ==
''Erin Walker, Bruce M. McLaren, Nikol Rummel, and Ken Koedinger''

=== Summary Tables ===
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''PIs''' || Bruce McLaren, Nikol Rummel
|-
| '''Other Contributers''' ||
* Graduate Student: Erin Walker
* Staff: Jonathan Steinhart
|}
 
'' Pre Study 1''
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''Study Start Date''' || 05/06
|-
| '''Study End Date''' || 06/06
|-
| '''LearnLab Site''' || CWCTC
|-
| '''LearnLab Course''' || Algebra
|-
| '''Number of Students''' || ''N'' = 14
|-
| '''Average # of hours per participant''' || 3 hrs.
|}
 

'' Pre Study 2''
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''Study Start Date''' || 11/06
|-
| '''Study End Date''' || 11/06
|-
| '''LearnLab Site''' || CWCTC
|-
| '''LearnLab Course''' || Algebra
|-
| '''Number of Students''' || ''N'' = 20
|-
| '''Average # of hours per participant''' || 3 hrs.
|}
 

'' Full Study''
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''Study Start Date''' || 04/07
|-
| '''Study End Date''' || 04/07
|-
| '''LearnLab Site''' || CWCTC
|-
| '''LearnLab Course''' || Algebra
|-
| '''Number of Students''' || ''N'' = 70
|-
| '''Average # of hours per participant''' || 3 hrs.
|}
 

=== Abstract ===
In this project, the Algebra I Cognitive Tutor is extended to a [[collaborative learning environment]]: Instead of the computer tutoring a student, students tutor each other, and the [[cognitive tutor]] provides collaborative support. As research has shown, collaborative problem solving and learning has the potential to increase elaboration on the learning content. However, students are not always able to effectively meet the challenges of a collaborative setting. To ensure that students capitalize on collaborative problem solving with the Tutor, a peer tutoring [[collaboration scripts|collaboration script]] was developed that guides their interaction and prompts fruitful [[collaboration]].

In the baseline version of the script (without cognitive tutor support), students in the same class are seated at different computers and take turs tutoring each other. First, they prepare to tutor by solving problems using the regular cognitive tutor. Then, they tutor each other by marking each other right and wrong, monitoring their partner's skills, and give each other hints and feedback. After preparing and tutoring, they reflect on the problems and their interaction, engaging in [[metacognition|metacognitive activities]]. We have expanded the script to include intelligent cognitive support of the peer tutoring, and hope to ultimately expand it to include intelligent collaborative support.

To assess the effectiveness of the script on [[robust learning]], we are conducting a series of [[in vivo experiment]]s in the Algebra LearnLab. In an initial, small scale study (pre study) that served to establish basic effects and to test the procedure in a classroom setting, we compared peer tutoring with reflection elements to peer tutoring without reflection elements. In a second study that served to explore the effects of cognitive tutoring of peer tutoring, we compared peer tutoring plus cognitive tutoring to peer tutoring. In the full study, we will compare these collaborative conditions to an individual condition to assess the effect of the collaborative Tutor extension to regular Tutor use.

=== Background and Significance ===

In our project, we combined two different [[instructional method]]s both of which have been shown to improve students’ learning in mathematics: Learning with intelligent tutoring systems (Koedinger, Anderson, Hadley, & Mark, 1997) and collaborative problem solving (Berg, 1993). The [[Cognitive tutor|Cognitive Tutor]] Algebra that was used in our study is a tutor for mathematics instruction at the high school level. Students learn with the Tutor during part of the regular classroom sessions. The Tutor's main features are immediate error feedback, the possibility to ask for a hint when encountering impasses, and knowledge tracing, i.e. the Tutor creates and updates a model of the student’s knowledge and selects new problems tailored to the student’s knowledge level. Although several studies have proven its effectiveness, students do not always benefit from learning with the Tutor. First, because the Tutor places emphasis on learning [[procedural|procedural problem solving skills]], yet a deep understanding of underlying mathematical [[conceptual knowledge|concepts]] is not necessarily achieved (Anderson, Corbett, Koedinger, & Pelletier, 1995). Second, students do not always make good use of the learning opportunities provided by the Cognitive Tutor (e.g. [[help abuse]], see Aleven, McLaren, Roll, & Koedinger, 2004; [[game the system|gaming the system]], Baker, Corbett, & Koedinger, 2004). So far, the Cognitive Tutor has been used in an individual learning setting only. However, as research on collaborative learning has shown, collaboration can yield elaboration of learning content (Teasley, 1995), thus this could be a promising approach to reduce the Tutor’s shortcomings. On the other hand, students are not always able to effectively meet the challenges of a collaborative setting (Rummel & Spada, 2005). [[Collaboration scripts]] have proven effective in helping people meet the challenges encountered when learning or working collaboratively (Kollar, Fischer, & Hesse, in press).

Combining collaborative activities with intelligent tutoring might combine the benefits of both approaches. Collaborative interaction could augment the effects of an existing ITS by adding deeper interaction, and intelligent tutoring support could improve the quality of student collaboration by providing guidance to students as they attempt to follow a collaboration script. One way to incorporate collaboration into an ITS is to place the intelligent tutoring system in the role of one of the collaborators and have a student interact with this agent. For example, Biswas, Schwartz, Leelawong, Vye, and the TAG-V (2005) have the student collaborate with an agent (“Betty”) in a peer tutoring scenario. The student teaches Betty about ecosystems with the help of a second mentoring agent (“Mr. Davis”), and it has been shown that a student who takes the role of a teacher in this system benefits more than a student who takes the role of a learner in a parallel traditional tutoring system. However, this system does not yet support natural language interaction between the agents involved, and therefore does not take full advantage of important collaborative interaction mechanisms that can increase learning. Another way to combine collaboration and tutoring is to use the ITS to structure and tutor the student collaboration, by connecting collaborative tools with ITS components (see Harrer, McLaren, Walker, Bollen, & Sewall, 2006), and developing a model of good and ineffective collaboration in order to provide on-line hints and feedback (see Soller, 2001). Most research into computer tutoring of the collaboration between two students is still in a preliminary stage.

We integrate collaborative learning with an ITS using a peer tutoring framework. Similar to Biswas et al. (2005), we put the student in the role of the tutor, and intend to have the cognitive elements of the tutoring supported by another agent. However, unlike Biswas et al., (2005), the teachable agent is a real student. Therefore, we have intelligent tutoring components monitoring and providing feedback on student collaboration in addition to student cognition, like Harrer et al. (2006). Our ultimate goal is to allow students to tutor each other through the interface of an ITS, supported by both cognitive and collaborative tutoring.

=== Glossary ===
See [[:Category:Peer Tutoring|Peer Tutoring Glossary]]

=== Research question ===
Does [[collaboration]] – and in particular [[collaboration scripts|scripted collaboration]] – improve students’ [[robust learning]] in the domain of algebra?

Do [[instructional method|instructional activities]] that involve three agents (a student, a peer tutor, and a computer tutor) increase [[robust learning]] compared to instructional activities that involve two agents (student and peer tutor, student and computer tutor)?

=== Independent variables ===
We vary the [[agent]]s involved in the interaction using the [[instructional method|mode of instruction]]. There are four conditions:
* Individual condition: Students solved problems with the Algebra I Cognitive Tutor in the regular fashion (i.e. computer program as additional [[agent]]).
* Peer tutoring condition: Students tutored each other using the cognitive tutor interface (i.e., another peer as additional [[agent]]). They went through two phases:
** a preparation phase, where students prepared to tutor while solving problems with the Algebra I Cognitive Tutor in the regular fashion
** a collaboration phase, where students take turns tutoring each other. Peer tutors can mark answers right or wrong, monitor their partner's skills, and give hints and feedback
* Peer tutoring plus reflection: As in the peer tutoring condition, students go through a preparation phase and then tutor each other in the collaboration phase. However, students also go through a series of [[metacognition|metacognitive]] self-reflection activities to encourage them to engage in [[elaborative interaction]]
* Peer tutoring plus cognitive tutoring: Students participate in the peer tutoring plus reflection condition, but during the collaboration phase, peer tutors receive cognitive tutor hints and feedback (i.e., a peer and computer program as additional [[agent]]s). This feedback contains two components:
** a collaborative prompt that encourages students to interact with each other
** a cognitive hint targeted at the particular mistake the peer tutee is making

=== Hypothesis ===
1. We expect the collaborative conditions to outperform the individual conditions in measures of robust learning of the algebra content.

2. We epect the peer tutoring plus cognitive tutoring condition to outperform the peer tutoring alone conditions in measures of robust learning of the algebra content

=== Dependent variables ===
* ''Near [[transfer]], immediate'': During training, student progress on training problems are be analyzed.
* ''Near [[transfer]], [[long-term retention|retention]]'': Students a are given a posttest immediately after the study on isomorphic problems
* ''Far [[transfer]]'': This paper and pencil test assessed students' understanding of the main mathematical concepts from the learning phase. The [[conceptual tasks]] students had to solve tapped the same knowledge components as the problems in instruction; however, the problems where non-isomorphic to those in the instruction, thus demanded students to flexibly apply their knowledge to problems with a new format.
* ''Acceleration of future learning'': Student learning on future equation solving units will be measured.

To compare [[collaboration skill]]s of students, we will be conducting an analysis of student dialogues durng the learning phase.

=== Explanation ===
This study is part of the [[Interactive_Communication|Interactive Communication cluster]], and its hypothesis is a specialization of the IC cluster’s central hypothesis. We assume that the instruction is in or above the student’s ZPD. Here is what we expect to happen in the three conditions:

* In the peer tutoring condition, the peer tutor’s help should increase learning compared to a student learning with the cognitive tutor. However, the student may make too many errors and/or require too much communication with the second agent. The peer tutor may not be able to provide the student with the help he/she needs. Learning will not be optimized.
* In the cognitive+peer tutoring condition, the peer tutor’s help should provide the additional benefits of interactive communication, and the cognitive tutor’s help should ensure that both the student and the peer tutor get the problem solving assistance they need. The support of the cognitive tutor and interaction with the student will also increase the peer tutor’s learning.
The target path will have the students learning-by-doing and learning-by-teaching, with the help of the computer agent so students do not make too many errors or require too much communication.

=== Findings ===
'''Pre-study 1:'''

Our first study compared the peer tutoring to peer tutoring plus reflection conditions. Using pretest and posttest scores, we conducted a two-way (condition x test-time) repeated-measure ANOVA, with test-time as the repeated measure. Posttest scores were significantly higher than pretest scores in both the tutoring and the tutoring+reflection condition (F (1,12) = 15.25, p < .002, η² = 0.56), but there were no significant differences between conditions, and no interaction (see Table 2). To further examine what occurred during the collaboration phase we turned to log data and notes from classroom observation. During peer tutoring, students appeared engaged, and did exhibit many of the positive collaborative behaviors that we were attempting to encourage with our script and that have been shown to correlate with knowledge construction and self-reflection. However, we observed that peer tutors struggled to provide tutees with answers, and did not connect the preparation that they had done with the collaboration phase. For instance, they often did not consult their answer printouts when they did not know the next problem step and thus had to rely on teacher assistance to solve a problem. As a result, tutees skipped problems without completing them correctly. This undesirable behavior differed between the two conditions. Students in the tutoring condition attempted more problems than students in the tutoring+reflection condition, and appeared to complete more problems as well. The average number of problems completed by dyads in the tutoring+reflection condition was low; students in this group took an average of 11 minutes to complete a single problem, compared to a 6 minute average in the tutoring condition. Students in the tutoring condition tended to skip problems they could not solve, completing less than 60% of the problems they attempted. Immediately before skipping a problem, students would generally state their inability to solve it, “I don’t know how to do this one,“ or their lack of motivation, “Just do something and I’ll agree or something.” If students skip problems, they may not learn how to solve difficult problems. However, if they do not complete many problems, they may not be sufficiently exposed to all the skills involved in the unit, and will be given fewer opportunities to master them.

'''Results for Pre-study 2 and Study 1 coming soon.'''

=== Explanation ===
This study is part of the [[Interactive_Communication|Interactive Communication cluster]], and its hypothesis is a specialization of the IC cluster’s central hypothesis. According to the IC cluster’s hypothesis, instruction that yields [[robust learning]] should be designed to have the right target paths. Furthermore, it should improve the students’ path choice in the [[learning event space]], i.e., increasing the probability that students take correct paths and decreasing the probability that students take alternative paths.

By enhancing the Cognitive Tutor with collaboration, the study tried to reach these two goals in two steps:

* ''Individual condition'': The existing Cognitive Tutor already offers correct learning paths. However, individual learning on the Cognitive Tutor has several shortcomings. For example, students don’t have the opportunity to reflect on the underlying mathematical [[conceptual knowledge|concepts]] in natural language interaction in order to gain a deeper understanding. In addition, students do not always use the offered learning paths (as the opportunity to reflect on on-demand hints and error feedback) effectively, thus often do not engage in [[sense making]] processes
* ''Peer tutoring condition (1st step)'': By enhancing the Algebra I Cognitive Tutor to be a collaborative learning environment, we added a learning resource – the learning partner. This adds further correct learning paths to the [[learning event space]], for instance, learning by giving explanations, the possibility to request help, and learning by knowledge co-construction. However, similarly to the learning paths in an individual setting, students do not always capitalize on these learning opportunities. Further, if students lack the necessary expertise to tutor, they will not master the desired knowledge components, even if they take a correct learning path.
* ''Peer plus cognitive tutoring condition (2nd step)'': To increase the probability that students are mastering the correct knowledge components, the cognitive tutor helps peer tutors interact with their partners. Students use both the peer tutor and the cognitive tutor as learning resources.

=== Annotated bibliography ===
* Walker, E., McLaren, B. M., Rummel, N., and Koedinger, K. R. Who Says Three's a Crowd? Using a Cognitive Tutor to Support Peer Tutoring. To appear in the Proceedings of the 13th International Conference on Artificial Intelligence and Education. 2007.
* Walker, E., Rummel, N., McLaren, B. M. & Koedinger, K. R. The Student Becomes the Master: Integrating Peer Tutoring with Cognitive Tutoring. Short paper at the Conference on Computer Supported Collaborative Learning (CSCL-07). Rutgers University, July 16-21, 2007.
* Walker, E., Koedinger, K., McLaren, B. M., & Rummel, N. (2006). Cognitive tutors as research platforms: Extending an established tutoring system for collaborative and metacognitive experimentation. ''Lecture Notes in Computer Science, Volume 4053/2006. Proceedings of the 8th International Conference on Intelligent Tutoring Systems'' (pp. 207-216). Berlin: Springer
* Walker, E. (2005). Mutual peer tutoring: A collaborative addition to the Algebra-1 Cognitive Tutor. Paper presented at the 12th International Conference on Artificial Intelligence and Education (AIED-05, Young Researchers Track), July, 2005, Amsterdam, the Netherlands.

=== References ===

* Aleven, V., McLaren, B., Roll, I., & Koedinger, K. R. (2004). Toward tutoring help seeking: Applying cognitive modelling to meta-cognitive skills. In J. C. Lester, R. M. Vicari & F. Paraguaçu (Eds.), Proceedings of Seventh International Conference on Intelligent Tutoring Systems, ITS 2004 (pp. 227-239). Berlin: Springer.
* Anderson, J. R., Corbett, A. T., Koedinger, K. R., & Pelletier, R. (1995). Cognitive tutors: Lessons learned. Journal of the Learning Sciences, 4(2), 167-207.
* Baker, R. S., Corbett, A. T., & Koedinger, K. R. (2004). Detecting student misuse of intelligent tutoring systems. Paper presented at the Proceedings of the 7th International Conference on Intelligent Tutoring Systems.
* Berg, K. F. (1993). Structured cooperative learning and achievement in a high school mathematics class. Paper presented at the Annual Meeting of the American Educational Research Association, Atlanta, GA.
* Biswas, G., Schwartz, D. L., Leelawong, K., Vye, N., & TAG-V. (2005). Learning by teaching: A new agent paradigm for educational software. Applied Artificial Intelligence, 19, 363–392.
* Carmien, S., Kollar, I., Fischer, G. & Fischer, F. (2006). The interplay of internal and external scripts. In F. Fischer, I. Kollar, H. Mandl &, J. Haake, Scripting computer-supported communication of knowledge. Cognitive, computational, and educational perspectives (pp. 289-311). New York: Springer.
* Fantuzzo, J. W., Riggio, R. E., Connelly, S., & Dimeff, L. A. (1992). Effects of reciprocal peer tutoring on mathematics and school adjustment: A componential analysis. Journal of Educational Psychology, 84(3), 331-339.
* Fuchs, L.S., Fuchs, D., Prentice, K., Burch, M., Hamlett, C.L., Owen, R., & Schroeter, K. (2003). Enhancing third-grade students' mathematical problem solving with self-regulated learning strategies. Journal of Educational Psychology, 95(2), 306-315.
* Harrer, A., McLaren, B. M., Walker, E., Bollen, L., and Sewall, J. (2006). Creating Cognitive Tutors for Collaborative Learning: Steps Toward Realization. User Modeling and User-Adapted Interaction: The Journal of Personalization Research (UMUAI) (2006) 16: 175-209.
* Johnson, D. W. and Johnson, R. T. (1990). Cooperative learning and achievement. In S. Sharan (Ed.), Cooperative learning: Theory and research (pp. 23-37). New York: Praeger.
* King, A., Staffieri, A., & Adelgais, A. (1998). Mutual peer tutoring: Effects of structuring tutorial interaction to scaffold peer learning. Journal of Educational Psychology, 90, 134-152.
* Koedinger, K. R., Anderson, J. R., Hadley, W. H., & Mark, M. A. (1997). Intelligent tutoring goes to school in the big city. International Journal of Artificial Intelligence in Education, 8, 30-43.
* Kollar, I., Fischer, F., & Hesse, F. W. (in press). Collaboration scripts - a conceptual analysis. Educational Psychology Review.
* Ploetzner, R., Dillenbourg, P., Preier, M.,&Traum, D. (1999). Learning by explaining to oneself and to others. In P. Dillenbourg (Ed.), Collaborative learning. Cognitive and computational approaches (pp. 103–121). Amsterdam: Pergamon.
* Ritter, S., Blessing, S. B., & Hadley, W. S. (2002). SBIR Phase I Final Report 2002. Department of Education. Department of Education RFP ED: 84-305S.
* Rummel, N., & Spada, H. (2005). Learning to Collaborate: An Instructional Approach to Promoting Collaborative Problem Solving in Computer-Mediated Settings. Journal of the Learning Sciences, 14(2), 201-241.
* Soller, A.L. (2001). Supporting Social Interaction in an Intelligent Collaborative Learning System. International Journal of Artificial Intelligence in Education, 12, 40-62.
* Teasley, S. D. (1995). The role of talk in children's peer collaborations. Developmental Psychology, 31(2), 207-220.
* Webb, N.M., Troper, J.D., & Fall, R. (1995). Constructive activity and learning in collaborative small groups. Journal of Educational Psychology, 87, 406-423.

Walker A Peer Tutoring Addition

2007-03-30T22:21:19Z

Erin-Walker: /* Annotated bibliography */

== Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition ==
''Erin Walker, Bruce M. McLaren, Nikol Rummel, and Ken Koedinger''

=== Summary Tables ===
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''PIs''' || Bruce McLaren, Nikol Rummel
|-
| '''Other Contributers''' ||
* Graduate Student: Erin Walker
* Staff: Jonathan Steinhart
|}
 
'' Pre Study 1''
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''Study Start Date''' || 05/06
|-
| '''Study End Date''' || 06/06
|-
| '''LearnLab Site''' || CWCTC
|-
| '''LearnLab Course''' || Algebra
|-
| '''Number of Students''' || ''N'' = 14
|-
| '''Average # of hours per participant''' || 3 hrs.
|}
 

'' Pre Study 2''
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''Study Start Date''' || 11/06
|-
| '''Study End Date''' || 11/06
|-
| '''LearnLab Site''' || CWCTC
|-
| '''LearnLab Course''' || Algebra
|-
| '''Number of Students''' || ''N'' = 20
|-
| '''Average # of hours per participant''' || 3 hrs.
|}
 

'' Full Study''
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''Study Start Date''' || 04/07
|-
| '''Study End Date''' || 04/07
|-
| '''LearnLab Site''' || CWCTC
|-
| '''LearnLab Course''' || Algebra
|-
| '''Number of Students''' || ''N'' = 70
|-
| '''Average # of hours per participant''' || 3 hrs.
|}
 

=== Abstract ===
In this project, the Algebra I Cognitive Tutor is extended to a [[collaborative learning environment]]: Instead of the computer tutoring a student, students tutor each other, and the [[cognitive tutor]] provides collaborative support. As research has shown, collaborative problem solving and learning has the potential to increase elaboration on the learning content. However, students are not always able to effectively meet the challenges of a collaborative setting. To ensure that students capitalize on collaborative problem solving with the Tutor, a peer tutoring [[collaboration scripts|collaboration script]] was developed that guides their interaction and prompts fruitful [[collaboration]].

In the baseline version of the script (without cognitive tutor support), students in the same class are seated at different computers and take turs tutoring each other. First, they prepare to tutor by solving problems using the regular cognitive tutor. Then, they tutor each other by marking each other right and wrong, monitoring their partner's skills, and give each other hints and feedback. After preparing and tutoring, they reflect on the problems and their interaction, engaging in [[metacognition|metacognitive activities]]. We have expanded the script to include intelligent cognitive support of the peer tutoring, and hope to ultimately expand it to include intelligent collaborative support.

To assess the effectiveness of the script on [[robust learning]], we are conducting a series of [[in vivo experiment]]s in the Algebra LearnLab. In an initial, small scale study (pre study) that served to establish basic effects and to test the procedure in a classroom setting, we compared peer tutoring with reflection elements to peer tutoring without reflection elements. In a second study that served to explore the effects of cognitive tutoring of peer tutoring, we compared peer tutoring plus cognitive tutoring to peer tutoring. In the full study, we will compare these collaborative conditions to an individual condition to assess the effect of the collaborative Tutor extension to regular Tutor use.

=== Background and Significance ===

In our project, we combined two different [[instructional method]]s both of which have been shown to improve students’ learning in mathematics: Learning with intelligent tutoring systems (Koedinger, Anderson, Hadley, & Mark, 1997) and collaborative problem solving (Berg, 1993). The [[Cognitive tutor|Cognitive Tutor]] Algebra that was used in our study is a tutor for mathematics instruction at the high school level. Students learn with the Tutor during part of the regular classroom sessions. The Tutor's main features are immediate error feedback, the possibility to ask for a hint when encountering impasses, and knowledge tracing, i.e. the Tutor creates and updates a model of the student’s knowledge and selects new problems tailored to the student’s knowledge level. Although several studies have proven its effectiveness, students do not always benefit from learning with the Tutor. First, because the Tutor places emphasis on learning [[procedural|procedural problem solving skills]], yet a deep understanding of underlying mathematical [[conceptual knowledge|concepts]] is not necessarily achieved (Anderson, Corbett, Koedinger, & Pelletier, 1995). Second, students do not always make good use of the learning opportunities provided by the Cognitive Tutor (e.g. [[help abuse]], see Aleven, McLaren, Roll, & Koedinger, 2004; [[game the system|gaming the system]], Baker, Corbett, & Koedinger, 2004). So far, the Cognitive Tutor has been used in an individual learning setting only. However, as research on collaborative learning has shown, collaboration can yield elaboration of learning content (Teasley, 1995), thus this could be a promising approach to reduce the Tutor’s shortcomings. On the other hand, students are not always able to effectively meet the challenges of a collaborative setting (Rummel & Spada, 2005). [[Collaboration scripts]] have proven effective in helping people meet the challenges encountered when learning or working collaboratively (Kollar, Fischer, & Hesse, in press).

Combining collaborative activities with intelligent tutoring might combine the benefits of both approaches. Collaborative interaction could augment the effects of an existing ITS by adding deeper interaction, and intelligent tutoring support could improve the quality of student collaboration by providing guidance to students as they attempt to follow a collaboration script. One way to incorporate collaboration into an ITS is to place the intelligent tutoring system in the role of one of the collaborators and have a student interact with this agent. For example, Biswas, Schwartz, Leelawong, Vye, and the TAG-V (2005) have the student collaborate with an agent (“Betty”) in a peer tutoring scenario. The student teaches Betty about ecosystems with the help of a second mentoring agent (“Mr. Davis”), and it has been shown that a student who takes the role of a teacher in this system benefits more than a student who takes the role of a learner in a parallel traditional tutoring system. However, this system does not yet support natural language interaction between the agents involved, and therefore does not take full advantage of important collaborative interaction mechanisms that can increase learning. Another way to combine collaboration and tutoring is to use the ITS to structure and tutor the student collaboration, by connecting collaborative tools with ITS components (see Harrer, McLaren, Walker, Bollen, & Sewall, 2006), and developing a model of good and ineffective collaboration in order to provide on-line hints and feedback (see Soller, 2001). Most research into computer tutoring of the collaboration between two students is still in a preliminary stage.

We integrate collaborative learning with an ITS using a peer tutoring framework. Similar to Biswas et al. (2005), we put the student in the role of the tutor, and intend to have the cognitive elements of the tutoring supported by another agent. However, unlike Biswas et al., (2005), the teachable agent is a real student. Therefore, we have intelligent tutoring components monitoring and providing feedback on student collaboration in addition to student cognition, like Harrer et al. (2006). Our ultimate goal is to allow students to tutor each other through the interface of an ITS, supported by both cognitive and collaborative tutoring.

=== Glossary ===
See [[:Category:Peer Tutoring|Peer Tutoring Glossary]]

=== Research question ===
Does [[collaboration]] – and in particular [[collaboration scripts|scripted collaboration]] – improve students’ [[robust learning]] in the domain of algebra?

Do [[instructional method|instructional activities]] that involve three agents (a student, a peer tutor, and a computer tutor) increase [[robust learning]] compared to instructional activities that involve two agents (student and peer tutor, student and computer tutor)?

=== Independent variables ===
We vary the [[agent]]s involved in the interaction using the [[instructional method|mode of instruction]]. There are four conditions:
* Individual condition: Students solved problems with the Algebra I Cognitive Tutor in the regular fashion (i.e. computer program as additional [[agent]]).
* Peer tutoring condition: Students tutored each other using the cognitive tutor interface (i.e., another peer as additional [[agent]]). They went through two phases:
** a preparation phase, where students prepared to tutor while solving problems with the Algebra I Cognitive Tutor in the regular fashion
** a collaboration phase, where students take turns tutoring each other. Peer tutors can mark answers right or wrong, monitor their partner's skills, and give hints and feedback
* Peer tutoring plus reflection: As in the peer tutoring condition, students go through a preparation phase and then tutor each other in the collaboration phase. However, students also go through a series of [[metacognition|metacognitive]] self-reflection activities to encourage them to engage in [[elaborative interaction]]
* Peer tutoring plus cognitive tutoring: Students participate in the peer tutoring plus reflection condition, but during the collaboration phase, peer tutors receive cognitive tutor hints and feedback (i.e., a peer and computer program as additional [[agent]]s). This feedback contains two components:
** a collaborative prompt that encourages students to interact with each other
** a cognitive hint targeted at the particular mistake the peer tutee is making

=== Hypothesis ===
1. We expect the collaborative conditions to outperform the individual conditions in measures of robust learning of the algebra content.

2. We epect the peer tutoring plus cognitive tutoring condition to outperform the peer tutoring alone conditions in measures of robust learning of the algebra content

=== Dependent variables ===
* ''Near [[transfer]], immediate'': During training, student progress on training problems are be analyzed.
* ''Near [[transfer]], [[long-term retention|retention]]'': Students a are given a posttest immediately after the study on isomorphic problems
* ''Far [[transfer]]'': This paper and pencil test assessed students' understanding of the main mathematical concepts from the learning phase. The [[conceptual tasks]] students had to solve tapped the same knowledge components as the problems in instruction; however, the problems where non-isomorphic to those in the instruction, thus demanded students to flexibly apply their knowledge to problems with a new format.
* ''Acceleration of future learning'': Student learning on future equation solving units will be measured.

To compare [[collaboration skill]]s of students, we will be conducting an analysis of student dialogues durng the learning phase.

=== Explanation ===
This study is part of the [[Interactive_Communication|Interactive Communication cluster]], and its hypothesis is a specialization of the IC cluster’s central hypothesis. We assume that the instruction is in or above the student’s ZPD. Here is what we expect to happen in the three conditions:

* In the peer tutoring condition, the peer tutor’s help should increase learning compared to a student learning with the cognitive tutor. However, the student may make too many errors and/or require too much communication with the second agent. The peer tutor may not be able to provide the student with the help he/she needs. Learning will not be optimized.
* In the cognitive+peer tutoring condition, the peer tutor’s help should provide the additional benefits of interactive communication, and the cognitive tutor’s help should ensure that both the student and the peer tutor get the problem solving assistance they need. The support of the cognitive tutor and interaction with the student will also increase the peer tutor’s learning.
The target path will have the students learning-by-doing and learning-by-teaching, with the help of the computer agent so students do not make too many errors or require too much communication.

=== Findings ===
'''Pre-study 1:'''

Our first study compared the peer tutoring to peer tutoring plus reflection conditions. Using pretest and posttest scores, we conducted a two-way (condition x test-time) repeated-measure ANOVA, with test-time as the repeated measure. Posttest scores were significantly higher than pretest scores in both the tutoring and the tutoring+reflection condition (F (1,12) = 15.25, p < .002, η² = 0.56), but there were no significant differences between conditions, and no interaction (see Table 2). To further examine what occurred during the collaboration phase we turned to log data and notes from classroom observation. During peer tutoring, students appeared engaged, and did exhibit many of the positive collaborative behaviors that we were attempting to encourage with our script and that have been shown to correlate with knowledge construction and self-reflection. However, we observed that peer tutors struggled to provide tutees with answers, and did not connect the preparation that they had done with the collaboration phase. For instance, they often did not consult their answer printouts when they did not know the next problem step and thus had to rely on teacher assistance to solve a problem. As a result, tutees skipped problems without completing them correctly. This undesirable behavior differed between the two conditions. Students in the tutoring condition attempted more problems than students in the tutoring+reflection condition, and appeared to complete more problems as well. The average number of problems completed by dyads in the tutoring+reflection condition was low; students in this group took an average of 11 minutes to complete a single problem, compared to a 6 minute average in the tutoring condition. Students in the tutoring condition tended to skip problems they could not solve, completing less than 60% of the problems they attempted. Immediately before skipping a problem, students would generally state their inability to solve it, “I don’t know how to do this one,“ or their lack of motivation, “Just do something and I’ll agree or something.” If students skip problems, they may not learn how to solve difficult problems. However, if they do not complete many problems, they may not be sufficiently exposed to all the skills involved in the unit, and will be given fewer opportunities to master them.

'''Results for Pre-study 2 and Study 1 coming soon.'''

=== Annotated bibliography ===
* Walker, E., McLaren, B. M., Rummel, N., and Koedinger, K. R. Who Says Three's a Crowd? Using a Cognitive Tutor to Support Peer Tutoring. To appear in the Proceedings of the 13th International Conference on Artificial Intelligence and Education. 2007.
* Walker, E., Rummel, N., McLaren, B. M. & Koedinger, K. R. The Student Becomes the Master: Integrating Peer Tutoring with Cognitive Tutoring. Short paper at the Conference on Computer Supported Collaborative Learning (CSCL-07). Rutgers University, July 16-21, 2007.
* Walker, E., Koedinger, K., McLaren, B. M., & Rummel, N. (2006). Cognitive tutors as research platforms: Extending an established tutoring system for collaborative and metacognitive experimentation. ''Lecture Notes in Computer Science, Volume 4053/2006. Proceedings of the 8th International Conference on Intelligent Tutoring Systems'' (pp. 207-216). Berlin: Springer
* Walker, E. (2005). Mutual peer tutoring: A collaborative addition to the Algebra-1 Cognitive Tutor. Paper presented at the 12th International Conference on Artificial Intelligence and Education (AIED-05, Young Researchers Track), July, 2005, Amsterdam, the Netherlands.

=== References ===

* Aleven, V., McLaren, B., Roll, I., & Koedinger, K. R. (2004). Toward tutoring help seeking: Applying cognitive modelling to meta-cognitive skills. In J. C. Lester, R. M. Vicari & F. Paraguaçu (Eds.), Proceedings of Seventh International Conference on Intelligent Tutoring Systems, ITS 2004 (pp. 227-239). Berlin: Springer.
* Anderson, J. R., Corbett, A. T., Koedinger, K. R., & Pelletier, R. (1995). Cognitive tutors: Lessons learned. Journal of the Learning Sciences, 4(2), 167-207.
* Baker, R. S., Corbett, A. T., & Koedinger, K. R. (2004). Detecting student misuse of intelligent tutoring systems. Paper presented at the Proceedings of the 7th International Conference on Intelligent Tutoring Systems.
* Berg, K. F. (1993). Structured cooperative learning and achievement in a high school mathematics class. Paper presented at the Annual Meeting of the American Educational Research Association, Atlanta, GA.
* Biswas, G., Schwartz, D. L., Leelawong, K., Vye, N., & TAG-V. (2005). Learning by teaching: A new agent paradigm for educational software. Applied Artificial Intelligence, 19, 363–392.
* Carmien, S., Kollar, I., Fischer, G. & Fischer, F. (2006). The interplay of internal and external scripts. In F. Fischer, I. Kollar, H. Mandl &, J. Haake, Scripting computer-supported communication of knowledge. Cognitive, computational, and educational perspectives (pp. 289-311). New York: Springer.
* Fantuzzo, J. W., Riggio, R. E., Connelly, S., & Dimeff, L. A. (1992). Effects of reciprocal peer tutoring on mathematics and school adjustment: A componential analysis. Journal of Educational Psychology, 84(3), 331-339.
* Fuchs, L.S., Fuchs, D., Prentice, K., Burch, M., Hamlett, C.L., Owen, R., & Schroeter, K. (2003). Enhancing third-grade students' mathematical problem solving with self-regulated learning strategies. Journal of Educational Psychology, 95(2), 306-315.
* Harrer, A., McLaren, B. M., Walker, E., Bollen, L., and Sewall, J. (2006). Creating Cognitive Tutors for Collaborative Learning: Steps Toward Realization. User Modeling and User-Adapted Interaction: The Journal of Personalization Research (UMUAI) (2006) 16: 175-209.
* Johnson, D. W. and Johnson, R. T. (1990). Cooperative learning and achievement. In S. Sharan (Ed.), Cooperative learning: Theory and research (pp. 23-37). New York: Praeger.
* King, A., Staffieri, A., & Adelgais, A. (1998). Mutual peer tutoring: Effects of structuring tutorial interaction to scaffold peer learning. Journal of Educational Psychology, 90, 134-152.
* Koedinger, K. R., Anderson, J. R., Hadley, W. H., & Mark, M. A. (1997). Intelligent tutoring goes to school in the big city. International Journal of Artificial Intelligence in Education, 8, 30-43.
* Kollar, I., Fischer, F., & Hesse, F. W. (in press). Collaboration scripts - a conceptual analysis. Educational Psychology Review.
* Ploetzner, R., Dillenbourg, P., Preier, M.,&Traum, D. (1999). Learning by explaining to oneself and to others. In P. Dillenbourg (Ed.), Collaborative learning. Cognitive and computational approaches (pp. 103–121). Amsterdam: Pergamon.
* Ritter, S., Blessing, S. B., & Hadley, W. S. (2002). SBIR Phase I Final Report 2002. Department of Education. Department of Education RFP ED: 84-305S.
* Rummel, N., & Spada, H. (2005). Learning to Collaborate: An Instructional Approach to Promoting Collaborative Problem Solving in Computer-Mediated Settings. Journal of the Learning Sciences, 14(2), 201-241.
* Soller, A.L. (2001). Supporting Social Interaction in an Intelligent Collaborative Learning System. International Journal of Artificial Intelligence in Education, 12, 40-62.
* Teasley, S. D. (1995). The role of talk in children's peer collaborations. Developmental Psychology, 31(2), 207-220.
* Webb, N.M., Troper, J.D., & Fall, R. (1995). Constructive activity and learning in collaborative small groups. Journal of Educational Psychology, 87, 406-423.

Walker A Peer Tutoring Addition

2007-03-30T22:18:54Z

Erin-Walker: /* References */

== Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition ==
''Erin Walker, Bruce M. McLaren, Nikol Rummel, and Ken Koedinger''

=== Summary Tables ===
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''PIs''' || Bruce McLaren, Nikol Rummel
|-
| '''Other Contributers''' ||
* Graduate Student: Erin Walker
* Staff: Jonathan Steinhart
|}
 
'' Pre Study 1''
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''Study Start Date''' || 05/06
|-
| '''Study End Date''' || 06/06
|-
| '''LearnLab Site''' || CWCTC
|-
| '''LearnLab Course''' || Algebra
|-
| '''Number of Students''' || ''N'' = 14
|-
| '''Average # of hours per participant''' || 3 hrs.
|}
 

'' Pre Study 2''
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''Study Start Date''' || 11/06
|-
| '''Study End Date''' || 11/06
|-
| '''LearnLab Site''' || CWCTC
|-
| '''LearnLab Course''' || Algebra
|-
| '''Number of Students''' || ''N'' = 20
|-
| '''Average # of hours per participant''' || 3 hrs.
|}
 

'' Full Study''
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''Study Start Date''' || 04/07
|-
| '''Study End Date''' || 04/07
|-
| '''LearnLab Site''' || CWCTC
|-
| '''LearnLab Course''' || Algebra
|-
| '''Number of Students''' || ''N'' = 70
|-
| '''Average # of hours per participant''' || 3 hrs.
|}
 

=== Abstract ===
In this project, the Algebra I Cognitive Tutor is extended to a [[collaborative learning environment]]: Instead of the computer tutoring a student, students tutor each other, and the [[cognitive tutor]] provides collaborative support. As research has shown, collaborative problem solving and learning has the potential to increase elaboration on the learning content. However, students are not always able to effectively meet the challenges of a collaborative setting. To ensure that students capitalize on collaborative problem solving with the Tutor, a peer tutoring [[collaboration scripts|collaboration script]] was developed that guides their interaction and prompts fruitful [[collaboration]].

In the baseline version of the script (without cognitive tutor support), students in the same class are seated at different computers and take turs tutoring each other. First, they prepare to tutor by solving problems using the regular cognitive tutor. Then, they tutor each other by marking each other right and wrong, monitoring their partner's skills, and give each other hints and feedback. After preparing and tutoring, they reflect on the problems and their interaction, engaging in [[metacognition|metacognitive activities]]. We have expanded the script to include intelligent cognitive support of the peer tutoring, and hope to ultimately expand it to include intelligent collaborative support.

To assess the effectiveness of the script on [[robust learning]], we are conducting a series of [[in vivo experiment]]s in the Algebra LearnLab. In an initial, small scale study (pre study) that served to establish basic effects and to test the procedure in a classroom setting, we compared peer tutoring with reflection elements to peer tutoring without reflection elements. In a second study that served to explore the effects of cognitive tutoring of peer tutoring, we compared peer tutoring plus cognitive tutoring to peer tutoring. In the full study, we will compare these collaborative conditions to an individual condition to assess the effect of the collaborative Tutor extension to regular Tutor use.

=== Background and Significance ===

In our project, we combined two different [[instructional method]]s both of which have been shown to improve students’ learning in mathematics: Learning with intelligent tutoring systems (Koedinger, Anderson, Hadley, & Mark, 1997) and collaborative problem solving (Berg, 1993). The [[Cognitive tutor|Cognitive Tutor]] Algebra that was used in our study is a tutor for mathematics instruction at the high school level. Students learn with the Tutor during part of the regular classroom sessions. The Tutor's main features are immediate error feedback, the possibility to ask for a hint when encountering impasses, and knowledge tracing, i.e. the Tutor creates and updates a model of the student’s knowledge and selects new problems tailored to the student’s knowledge level. Although several studies have proven its effectiveness, students do not always benefit from learning with the Tutor. First, because the Tutor places emphasis on learning [[procedural|procedural problem solving skills]], yet a deep understanding of underlying mathematical [[conceptual knowledge|concepts]] is not necessarily achieved (Anderson, Corbett, Koedinger, & Pelletier, 1995). Second, students do not always make good use of the learning opportunities provided by the Cognitive Tutor (e.g. [[help abuse]], see Aleven, McLaren, Roll, & Koedinger, 2004; [[game the system|gaming the system]], Baker, Corbett, & Koedinger, 2004). So far, the Cognitive Tutor has been used in an individual learning setting only. However, as research on collaborative learning has shown, collaboration can yield elaboration of learning content (Teasley, 1995), thus this could be a promising approach to reduce the Tutor’s shortcomings. On the other hand, students are not always able to effectively meet the challenges of a collaborative setting (Rummel & Spada, 2005). [[Collaboration scripts]] have proven effective in helping people meet the challenges encountered when learning or working collaboratively (Kollar, Fischer, & Hesse, in press).

Combining collaborative activities with intelligent tutoring might combine the benefits of both approaches. Collaborative interaction could augment the effects of an existing ITS by adding deeper interaction, and intelligent tutoring support could improve the quality of student collaboration by providing guidance to students as they attempt to follow a collaboration script. One way to incorporate collaboration into an ITS is to place the intelligent tutoring system in the role of one of the collaborators and have a student interact with this agent. For example, Biswas, Schwartz, Leelawong, Vye, and the TAG-V (2005) have the student collaborate with an agent (“Betty”) in a peer tutoring scenario. The student teaches Betty about ecosystems with the help of a second mentoring agent (“Mr. Davis”), and it has been shown that a student who takes the role of a teacher in this system benefits more than a student who takes the role of a learner in a parallel traditional tutoring system. However, this system does not yet support natural language interaction between the agents involved, and therefore does not take full advantage of important collaborative interaction mechanisms that can increase learning. Another way to combine collaboration and tutoring is to use the ITS to structure and tutor the student collaboration, by connecting collaborative tools with ITS components (see Harrer, McLaren, Walker, Bollen, & Sewall, 2006), and developing a model of good and ineffective collaboration in order to provide on-line hints and feedback (see Soller, 2001). Most research into computer tutoring of the collaboration between two students is still in a preliminary stage.

We integrate collaborative learning with an ITS using a peer tutoring framework. Similar to Biswas et al. (2005), we put the student in the role of the tutor, and intend to have the cognitive elements of the tutoring supported by another agent. However, unlike Biswas et al., (2005), the teachable agent is a real student. Therefore, we have intelligent tutoring components monitoring and providing feedback on student collaboration in addition to student cognition, like Harrer et al. (2006). Our ultimate goal is to allow students to tutor each other through the interface of an ITS, supported by both cognitive and collaborative tutoring.

=== Glossary ===
See [[:Category:Peer Tutoring|Peer Tutoring Glossary]]

=== Research question ===
Does [[collaboration]] – and in particular [[collaboration scripts|scripted collaboration]] – improve students’ [[robust learning]] in the domain of algebra?

Do [[instructional method|instructional activities]] that involve three agents (a student, a peer tutor, and a computer tutor) increase [[robust learning]] compared to instructional activities that involve two agents (student and peer tutor, student and computer tutor)?

=== Independent variables ===
We vary the [[agent]]s involved in the interaction using the [[instructional method|mode of instruction]]. There are four conditions:
* Individual condition: Students solved problems with the Algebra I Cognitive Tutor in the regular fashion (i.e. computer program as additional [[agent]]).
* Peer tutoring condition: Students tutored each other using the cognitive tutor interface (i.e., another peer as additional [[agent]]). They went through two phases:
** a preparation phase, where students prepared to tutor while solving problems with the Algebra I Cognitive Tutor in the regular fashion
** a collaboration phase, where students take turns tutoring each other. Peer tutors can mark answers right or wrong, monitor their partner's skills, and give hints and feedback
* Peer tutoring plus reflection: As in the peer tutoring condition, students go through a preparation phase and then tutor each other in the collaboration phase. However, students also go through a series of [[metacognition|metacognitive]] self-reflection activities to encourage them to engage in [[elaborative interaction]]
* Peer tutoring plus cognitive tutoring: Students participate in the peer tutoring plus reflection condition, but during the collaboration phase, peer tutors receive cognitive tutor hints and feedback (i.e., a peer and computer program as additional [[agent]]s). This feedback contains two components:
** a collaborative prompt that encourages students to interact with each other
** a cognitive hint targeted at the particular mistake the peer tutee is making

=== Hypothesis ===
1. We expect the collaborative conditions to outperform the individual conditions in measures of robust learning of the algebra content.

2. We epect the peer tutoring plus cognitive tutoring condition to outperform the peer tutoring alone conditions in measures of robust learning of the algebra content

=== Dependent variables ===
* ''Near [[transfer]], immediate'': During training, student progress on training problems are be analyzed.
* ''Near [[transfer]], [[long-term retention|retention]]'': Students a are given a posttest immediately after the study on isomorphic problems
* ''Far [[transfer]]'': This paper and pencil test assessed students' understanding of the main mathematical concepts from the learning phase. The [[conceptual tasks]] students had to solve tapped the same knowledge components as the problems in instruction; however, the problems where non-isomorphic to those in the instruction, thus demanded students to flexibly apply their knowledge to problems with a new format.
* ''Acceleration of future learning'': Student learning on future equation solving units will be measured.

To compare [[collaboration skill]]s of students, we will be conducting an analysis of student dialogues durng the learning phase.

=== Explanation ===
This study is part of the [[Interactive_Communication|Interactive Communication cluster]], and its hypothesis is a specialization of the IC cluster’s central hypothesis. We assume that the instruction is in or above the student’s ZPD. Here is what we expect to happen in the three conditions:

* In the peer tutoring condition, the peer tutor’s help should increase learning compared to a student learning with the cognitive tutor. However, the student may make too many errors and/or require too much communication with the second agent. The peer tutor may not be able to provide the student with the help he/she needs. Learning will not be optimized.
* In the cognitive+peer tutoring condition, the peer tutor’s help should provide the additional benefits of interactive communication, and the cognitive tutor’s help should ensure that both the student and the peer tutor get the problem solving assistance they need. The support of the cognitive tutor and interaction with the student will also increase the peer tutor’s learning.
The target path will have the students learning-by-doing and learning-by-teaching, with the help of the computer agent so students do not make too many errors or require too much communication.

=== Findings ===
'''Pre-study 1:'''

Our first study compared the peer tutoring to peer tutoring plus reflection conditions. Using pretest and posttest scores, we conducted a two-way (condition x test-time) repeated-measure ANOVA, with test-time as the repeated measure. Posttest scores were significantly higher than pretest scores in both the tutoring and the tutoring+reflection condition (F (1,12) = 15.25, p < .002, η² = 0.56), but there were no significant differences between conditions, and no interaction (see Table 2). To further examine what occurred during the collaboration phase we turned to log data and notes from classroom observation. During peer tutoring, students appeared engaged, and did exhibit many of the positive collaborative behaviors that we were attempting to encourage with our script and that have been shown to correlate with knowledge construction and self-reflection. However, we observed that peer tutors struggled to provide tutees with answers, and did not connect the preparation that they had done with the collaboration phase. For instance, they often did not consult their answer printouts when they did not know the next problem step and thus had to rely on teacher assistance to solve a problem. As a result, tutees skipped problems without completing them correctly. This undesirable behavior differed between the two conditions. Students in the tutoring condition attempted more problems than students in the tutoring+reflection condition, and appeared to complete more problems as well. The average number of problems completed by dyads in the tutoring+reflection condition was low; students in this group took an average of 11 minutes to complete a single problem, compared to a 6 minute average in the tutoring condition. Students in the tutoring condition tended to skip problems they could not solve, completing less than 60% of the problems they attempted. Immediately before skipping a problem, students would generally state their inability to solve it, “I don’t know how to do this one,“ or their lack of motivation, “Just do something and I’ll agree or something.” If students skip problems, they may not learn how to solve difficult problems. However, if they do not complete many problems, they may not be sufficiently exposed to all the skills involved in the unit, and will be given fewer opportunities to master them.

'''Results for Pre-study 2 and Study 1 coming soon.'''

=== Annotated bibliography ===
Walker, E., Rummel, N., McLaren, B. M. & Koedinger, K. R. (submitted). The Student Becomes the Master: Integrating Peer Tutoring with Cognitive Tutoring. Submitted to the conference on Computer Supported Collaborative Learning (CSCL-07). Rutgers University, July 16-21, 2007.

Walker, E., Koedinger, K., McLaren, B. M., & Rummel, N. (2006). Cognitive tutors as research platforms: Extending an established tutoring system for collaborative and metacognitive experimentation. ''Lecture Notes in Computer Science, Volume 4053/2006. Proceedings of the 8th International Conference on Intelligent Tutoring Systems'' (pp. 207-216). Berlin: Springer

Walker, E. (2005). Mutual peer tutoring: A collaborative addition to the Algebra-1 Cognitive Tutor. Paper presented at the 12th International Conference on Artificial Intelligence and Education (AIED-05, Young Researchers Track), July, 2005, Amsterdam, the Netherlands.

=== References ===

* Aleven, V., McLaren, B., Roll, I., & Koedinger, K. R. (2004). Toward tutoring help seeking: Applying cognitive modelling to meta-cognitive skills. In J. C. Lester, R. M. Vicari & F. Paraguaçu (Eds.), Proceedings of Seventh International Conference on Intelligent Tutoring Systems, ITS 2004 (pp. 227-239). Berlin: Springer.
* Anderson, J. R., Corbett, A. T., Koedinger, K. R., & Pelletier, R. (1995). Cognitive tutors: Lessons learned. Journal of the Learning Sciences, 4(2), 167-207.
* Baker, R. S., Corbett, A. T., & Koedinger, K. R. (2004). Detecting student misuse of intelligent tutoring systems. Paper presented at the Proceedings of the 7th International Conference on Intelligent Tutoring Systems.
* Berg, K. F. (1993). Structured cooperative learning and achievement in a high school mathematics class. Paper presented at the Annual Meeting of the American Educational Research Association, Atlanta, GA.
* Biswas, G., Schwartz, D. L., Leelawong, K., Vye, N., & TAG-V. (2005). Learning by teaching: A new agent paradigm for educational software. Applied Artificial Intelligence, 19, 363–392.
* Carmien, S., Kollar, I., Fischer, G. & Fischer, F. (2006). The interplay of internal and external scripts. In F. Fischer, I. Kollar, H. Mandl &, J. Haake, Scripting computer-supported communication of knowledge. Cognitive, computational, and educational perspectives (pp. 289-311). New York: Springer.
* Fantuzzo, J. W., Riggio, R. E., Connelly, S., & Dimeff, L. A. (1992). Effects of reciprocal peer tutoring on mathematics and school adjustment: A componential analysis. Journal of Educational Psychology, 84(3), 331-339.
* Fuchs, L.S., Fuchs, D., Prentice, K., Burch, M., Hamlett, C.L., Owen, R., & Schroeter, K. (2003). Enhancing third-grade students' mathematical problem solving with self-regulated learning strategies. Journal of Educational Psychology, 95(2), 306-315.
* Harrer, A., McLaren, B. M., Walker, E., Bollen, L., and Sewall, J. (2006). Creating Cognitive Tutors for Collaborative Learning: Steps Toward Realization. User Modeling and User-Adapted Interaction: The Journal of Personalization Research (UMUAI) (2006) 16: 175-209.
* Johnson, D. W. and Johnson, R. T. (1990). Cooperative learning and achievement. In S. Sharan (Ed.), Cooperative learning: Theory and research (pp. 23-37). New York: Praeger.
* King, A., Staffieri, A., & Adelgais, A. (1998). Mutual peer tutoring: Effects of structuring tutorial interaction to scaffold peer learning. Journal of Educational Psychology, 90, 134-152.
* Koedinger, K. R., Anderson, J. R., Hadley, W. H., & Mark, M. A. (1997). Intelligent tutoring goes to school in the big city. International Journal of Artificial Intelligence in Education, 8, 30-43.
* Kollar, I., Fischer, F., & Hesse, F. W. (in press). Collaboration scripts - a conceptual analysis. Educational Psychology Review.
* Ploetzner, R., Dillenbourg, P., Preier, M.,&Traum, D. (1999). Learning by explaining to oneself and to others. In P. Dillenbourg (Ed.), Collaborative learning. Cognitive and computational approaches (pp. 103–121). Amsterdam: Pergamon.
* Ritter, S., Blessing, S. B., & Hadley, W. S. (2002). SBIR Phase I Final Report 2002. Department of Education. Department of Education RFP ED: 84-305S.
* Rummel, N., & Spada, H. (2005). Learning to Collaborate: An Instructional Approach to Promoting Collaborative Problem Solving in Computer-Mediated Settings. Journal of the Learning Sciences, 14(2), 201-241.
* Soller, A.L. (2001). Supporting Social Interaction in an Intelligent Collaborative Learning System. International Journal of Artificial Intelligence in Education, 12, 40-62.
* Teasley, S. D. (1995). The role of talk in children's peer collaborations. Developmental Psychology, 31(2), 207-220.
* Webb, N.M., Troper, J.D., & Fall, R. (1995). Constructive activity and learning in collaborative small groups. Journal of Educational Psychology, 87, 406-423.

Walker A Peer Tutoring Addition

2007-03-30T22:14:41Z

Erin-Walker: /* Findings */

== Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition ==
''Erin Walker, Bruce M. McLaren, Nikol Rummel, and Ken Koedinger''

=== Summary Tables ===
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''PIs''' || Bruce McLaren, Nikol Rummel
|-
| '''Other Contributers''' ||
* Graduate Student: Erin Walker
* Staff: Jonathan Steinhart
|}
 
'' Pre Study 1''
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''Study Start Date''' || 05/06
|-
| '''Study End Date''' || 06/06
|-
| '''LearnLab Site''' || CWCTC
|-
| '''LearnLab Course''' || Algebra
|-
| '''Number of Students''' || ''N'' = 14
|-
| '''Average # of hours per participant''' || 3 hrs.
|}
 

'' Pre Study 2''
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''Study Start Date''' || 11/06
|-
| '''Study End Date''' || 11/06
|-
| '''LearnLab Site''' || CWCTC
|-
| '''LearnLab Course''' || Algebra
|-
| '''Number of Students''' || ''N'' = 20
|-
| '''Average # of hours per participant''' || 3 hrs.
|}
 

'' Full Study''
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''Study Start Date''' || 04/07
|-
| '''Study End Date''' || 04/07
|-
| '''LearnLab Site''' || CWCTC
|-
| '''LearnLab Course''' || Algebra
|-
| '''Number of Students''' || ''N'' = 70
|-
| '''Average # of hours per participant''' || 3 hrs.
|}
 

=== Abstract ===
In this project, the Algebra I Cognitive Tutor is extended to a [[collaborative learning environment]]: Instead of the computer tutoring a student, students tutor each other, and the [[cognitive tutor]] provides collaborative support. As research has shown, collaborative problem solving and learning has the potential to increase elaboration on the learning content. However, students are not always able to effectively meet the challenges of a collaborative setting. To ensure that students capitalize on collaborative problem solving with the Tutor, a peer tutoring [[collaboration scripts|collaboration script]] was developed that guides their interaction and prompts fruitful [[collaboration]].

In the baseline version of the script (without cognitive tutor support), students in the same class are seated at different computers and take turs tutoring each other. First, they prepare to tutor by solving problems using the regular cognitive tutor. Then, they tutor each other by marking each other right and wrong, monitoring their partner's skills, and give each other hints and feedback. After preparing and tutoring, they reflect on the problems and their interaction, engaging in [[metacognition|metacognitive activities]]. We have expanded the script to include intelligent cognitive support of the peer tutoring, and hope to ultimately expand it to include intelligent collaborative support.

To assess the effectiveness of the script on [[robust learning]], we are conducting a series of [[in vivo experiment]]s in the Algebra LearnLab. In an initial, small scale study (pre study) that served to establish basic effects and to test the procedure in a classroom setting, we compared peer tutoring with reflection elements to peer tutoring without reflection elements. In a second study that served to explore the effects of cognitive tutoring of peer tutoring, we compared peer tutoring plus cognitive tutoring to peer tutoring. In the full study, we will compare these collaborative conditions to an individual condition to assess the effect of the collaborative Tutor extension to regular Tutor use.

=== Background and Significance ===

In our project, we combined two different [[instructional method]]s both of which have been shown to improve students’ learning in mathematics: Learning with intelligent tutoring systems (Koedinger, Anderson, Hadley, & Mark, 1997) and collaborative problem solving (Berg, 1993). The [[Cognitive tutor|Cognitive Tutor]] Algebra that was used in our study is a tutor for mathematics instruction at the high school level. Students learn with the Tutor during part of the regular classroom sessions. The Tutor's main features are immediate error feedback, the possibility to ask for a hint when encountering impasses, and knowledge tracing, i.e. the Tutor creates and updates a model of the student’s knowledge and selects new problems tailored to the student’s knowledge level. Although several studies have proven its effectiveness, students do not always benefit from learning with the Tutor. First, because the Tutor places emphasis on learning [[procedural|procedural problem solving skills]], yet a deep understanding of underlying mathematical [[conceptual knowledge|concepts]] is not necessarily achieved (Anderson, Corbett, Koedinger, & Pelletier, 1995). Second, students do not always make good use of the learning opportunities provided by the Cognitive Tutor (e.g. [[help abuse]], see Aleven, McLaren, Roll, & Koedinger, 2004; [[game the system|gaming the system]], Baker, Corbett, & Koedinger, 2004). So far, the Cognitive Tutor has been used in an individual learning setting only. However, as research on collaborative learning has shown, collaboration can yield elaboration of learning content (Teasley, 1995), thus this could be a promising approach to reduce the Tutor’s shortcomings. On the other hand, students are not always able to effectively meet the challenges of a collaborative setting (Rummel & Spada, 2005). [[Collaboration scripts]] have proven effective in helping people meet the challenges encountered when learning or working collaboratively (Kollar, Fischer, & Hesse, in press).

Combining collaborative activities with intelligent tutoring might combine the benefits of both approaches. Collaborative interaction could augment the effects of an existing ITS by adding deeper interaction, and intelligent tutoring support could improve the quality of student collaboration by providing guidance to students as they attempt to follow a collaboration script. One way to incorporate collaboration into an ITS is to place the intelligent tutoring system in the role of one of the collaborators and have a student interact with this agent. For example, Biswas, Schwartz, Leelawong, Vye, and the TAG-V (2005) have the student collaborate with an agent (“Betty”) in a peer tutoring scenario. The student teaches Betty about ecosystems with the help of a second mentoring agent (“Mr. Davis”), and it has been shown that a student who takes the role of a teacher in this system benefits more than a student who takes the role of a learner in a parallel traditional tutoring system. However, this system does not yet support natural language interaction between the agents involved, and therefore does not take full advantage of important collaborative interaction mechanisms that can increase learning. Another way to combine collaboration and tutoring is to use the ITS to structure and tutor the student collaboration, by connecting collaborative tools with ITS components (see Harrer, McLaren, Walker, Bollen, & Sewall, 2006), and developing a model of good and ineffective collaboration in order to provide on-line hints and feedback (see Soller, 2001). Most research into computer tutoring of the collaboration between two students is still in a preliminary stage.

We integrate collaborative learning with an ITS using a peer tutoring framework. Similar to Biswas et al. (2005), we put the student in the role of the tutor, and intend to have the cognitive elements of the tutoring supported by another agent. However, unlike Biswas et al., (2005), the teachable agent is a real student. Therefore, we have intelligent tutoring components monitoring and providing feedback on student collaboration in addition to student cognition, like Harrer et al. (2006). Our ultimate goal is to allow students to tutor each other through the interface of an ITS, supported by both cognitive and collaborative tutoring.

=== Glossary ===
See [[:Category:Peer Tutoring|Peer Tutoring Glossary]]

=== Research question ===
Does [[collaboration]] – and in particular [[collaboration scripts|scripted collaboration]] – improve students’ [[robust learning]] in the domain of algebra?

Do [[instructional method|instructional activities]] that involve three agents (a student, a peer tutor, and a computer tutor) increase [[robust learning]] compared to instructional activities that involve two agents (student and peer tutor, student and computer tutor)?

=== Independent variables ===
We vary the [[agent]]s involved in the interaction using the [[instructional method|mode of instruction]]. There are four conditions:
* Individual condition: Students solved problems with the Algebra I Cognitive Tutor in the regular fashion (i.e. computer program as additional [[agent]]).
* Peer tutoring condition: Students tutored each other using the cognitive tutor interface (i.e., another peer as additional [[agent]]). They went through two phases:
** a preparation phase, where students prepared to tutor while solving problems with the Algebra I Cognitive Tutor in the regular fashion
** a collaboration phase, where students take turns tutoring each other. Peer tutors can mark answers right or wrong, monitor their partner's skills, and give hints and feedback
* Peer tutoring plus reflection: As in the peer tutoring condition, students go through a preparation phase and then tutor each other in the collaboration phase. However, students also go through a series of [[metacognition|metacognitive]] self-reflection activities to encourage them to engage in [[elaborative interaction]]
* Peer tutoring plus cognitive tutoring: Students participate in the peer tutoring plus reflection condition, but during the collaboration phase, peer tutors receive cognitive tutor hints and feedback (i.e., a peer and computer program as additional [[agent]]s). This feedback contains two components:
** a collaborative prompt that encourages students to interact with each other
** a cognitive hint targeted at the particular mistake the peer tutee is making

=== Hypothesis ===
1. We expect the collaborative conditions to outperform the individual conditions in measures of robust learning of the algebra content.

2. We epect the peer tutoring plus cognitive tutoring condition to outperform the peer tutoring alone conditions in measures of robust learning of the algebra content

=== Dependent variables ===
* ''Near [[transfer]], immediate'': During training, student progress on training problems are be analyzed.
* ''Near [[transfer]], [[long-term retention|retention]]'': Students a are given a posttest immediately after the study on isomorphic problems
* ''Far [[transfer]]'': This paper and pencil test assessed students' understanding of the main mathematical concepts from the learning phase. The [[conceptual tasks]] students had to solve tapped the same knowledge components as the problems in instruction; however, the problems where non-isomorphic to those in the instruction, thus demanded students to flexibly apply their knowledge to problems with a new format.
* ''Acceleration of future learning'': Student learning on future equation solving units will be measured.

To compare [[collaboration skill]]s of students, we will be conducting an analysis of student dialogues durng the learning phase.

=== Explanation ===
This study is part of the [[Interactive_Communication|Interactive Communication cluster]], and its hypothesis is a specialization of the IC cluster’s central hypothesis. We assume that the instruction is in or above the student’s ZPD. Here is what we expect to happen in the three conditions:

* In the peer tutoring condition, the peer tutor’s help should increase learning compared to a student learning with the cognitive tutor. However, the student may make too many errors and/or require too much communication with the second agent. The peer tutor may not be able to provide the student with the help he/she needs. Learning will not be optimized.
* In the cognitive+peer tutoring condition, the peer tutor’s help should provide the additional benefits of interactive communication, and the cognitive tutor’s help should ensure that both the student and the peer tutor get the problem solving assistance they need. The support of the cognitive tutor and interaction with the student will also increase the peer tutor’s learning.
The target path will have the students learning-by-doing and learning-by-teaching, with the help of the computer agent so students do not make too many errors or require too much communication.

=== Findings ===
'''Pre-study 1:'''

Our first study compared the peer tutoring to peer tutoring plus reflection conditions. Using pretest and posttest scores, we conducted a two-way (condition x test-time) repeated-measure ANOVA, with test-time as the repeated measure. Posttest scores were significantly higher than pretest scores in both the tutoring and the tutoring+reflection condition (F (1,12) = 15.25, p < .002, η² = 0.56), but there were no significant differences between conditions, and no interaction (see Table 2). To further examine what occurred during the collaboration phase we turned to log data and notes from classroom observation. During peer tutoring, students appeared engaged, and did exhibit many of the positive collaborative behaviors that we were attempting to encourage with our script and that have been shown to correlate with knowledge construction and self-reflection. However, we observed that peer tutors struggled to provide tutees with answers, and did not connect the preparation that they had done with the collaboration phase. For instance, they often did not consult their answer printouts when they did not know the next problem step and thus had to rely on teacher assistance to solve a problem. As a result, tutees skipped problems without completing them correctly. This undesirable behavior differed between the two conditions. Students in the tutoring condition attempted more problems than students in the tutoring+reflection condition, and appeared to complete more problems as well. The average number of problems completed by dyads in the tutoring+reflection condition was low; students in this group took an average of 11 minutes to complete a single problem, compared to a 6 minute average in the tutoring condition. Students in the tutoring condition tended to skip problems they could not solve, completing less than 60% of the problems they attempted. Immediately before skipping a problem, students would generally state their inability to solve it, “I don’t know how to do this one,“ or their lack of motivation, “Just do something and I’ll agree or something.” If students skip problems, they may not learn how to solve difficult problems. However, if they do not complete many problems, they may not be sufficiently exposed to all the skills involved in the unit, and will be given fewer opportunities to master them.

'''Results for Pre-study 2 and Study 1 coming soon.'''

=== Annotated bibliography ===
Walker, E., Rummel, N., McLaren, B. M. & Koedinger, K. R. (submitted). The Student Becomes the Master: Integrating Peer Tutoring with Cognitive Tutoring. Submitted to the conference on Computer Supported Collaborative Learning (CSCL-07). Rutgers University, July 16-21, 2007.

Walker, E., Koedinger, K., McLaren, B. M., & Rummel, N. (2006). Cognitive tutors as research platforms: Extending an established tutoring system for collaborative and metacognitive experimentation. ''Lecture Notes in Computer Science, Volume 4053/2006. Proceedings of the 8th International Conference on Intelligent Tutoring Systems'' (pp. 207-216). Berlin: Springer

Walker, E. (2005). Mutual peer tutoring: A collaborative addition to the Algebra-1 Cognitive Tutor. Paper presented at the 12th International Conference on Artificial Intelligence and Education (AIED-05, Young Researchers Track), July, 2005, Amsterdam, the Netherlands.

=== References ===

Biswas, G., Schwartz, D. L., Leelawong, K., Vye, N., & TAG-V. (2005). Learning by teaching: A new agent paradigm for educational software. Applied Artificial Intelligence, 19, 363–392.

Carmien, S., Kollar, I., Fischer, G. & Fischer, F. (2006). The interplay of internal and external scripts. In F. Fischer, I. Kollar, H. Mandl &, J. Haake, Scripting computer-supported communication of knowledge. Cognitive, computational, and educational perspectives (pp. 289-311). New York: Springer.

Fantuzzo, J. W., Riggio, R. E., Connelly, S., & Dimeff, L. A. (1992). Effects of reciprocal peer tutoring on mathematics and school adjustment: A componential analysis. Journal of Educational Psychology, 84(3), 331-339.

Fuchs, L.S., Fuchs, D., Prentice, K., Burch, M., Hamlett, C.L., Owen, R., & Schroeter, K. (2003). Enhancing third-grade students' mathematical problem solving with self-regulated learning strategies. Journal of Educational Psychology, 95(2), 306-315.

Harrer, A., McLaren, B. M., Walker, E., Bollen, L., and Sewall, J. (2006). Creating Cognitive Tutors for Collaborative Learning: Steps Toward Realization. User Modeling and User-Adapted Interaction: The Journal of Personalization Research (UMUAI) (2006) 16: 175-209.

Johnson, D. W. and Johnson, R. T. (1990). Cooperative learning and achievement. In S. Sharan (Ed.), Cooperative learning: Theory and research (pp. 23-37). New York: Praeger.

King, A., Staffieri, A., & Adelgais, A. (1998). Mutual peer tutoring: Effects of structuring tutorial interaction to scaffold peer learning. Journal of Educational Psychology, 90, 134-152.

Koedinger, K. R., Anderson, J. R., Hadley, W. H., & Mark, M. A. (1997). Intelligent tutoring goes to school in the big city. International Journal of Artificial Intelligence in Education, 8, 30-43.

Kollar, I., Fischer, F., & Hesse, F. W. (in press). Collaboration scripts - a conceptual analysis. Educational Psychology Review.

Ploetzner, R., Dillenbourg, P., Preier, M.,&Traum, D. (1999). Learning by explaining to oneself and to others. In P. Dillenbourg (Ed.), Collaborative learning. Cognitive and computational approaches (pp. 103–121). Amsterdam: Pergamon.

Ritter, S., Blessing, S. B., & Hadley, W. S. (2002). SBIR Phase I Final Report 2002. Department of Education. Department of Education RFP ED: 84-305S.

Rummel, N., & Spada, H. (2005). Learning to collaborate: An instructional approach to promoting collaborative problem-solving in computer-mediated settings. Journal of the Learning Sciences, 14(2), 201--241.

Soller, A.L. (2001). Supporting Social Interaction in an Intelligent Collaborative Learning System. International Journal of Artificial Intelligence in Education, 12, 40-62.

Teasley, S. D. (1995). The role of talk in children's peer collaborations. Developmental Psychology, 31(2), 207-220.

Walker, E., Koedinger, K. R., McLaren, B. M. and Rummel, N. Cognitive Tutors as Research Platforms: Extending an Established Tutoring System for Collaborative and Metacognitive Experimentation. In the Proceedings of the 8th International Conference on Intelligent Tutoring Systems, Jhongli, Taiwan, June 26-30, 2006.

Webb, N.M., Troper, J.D., & Fall, R. (1995). Constructive activity and learning in collaborative small groups. Journal of Educational Psychology, 87, 406-423.

Walker A Peer Tutoring Addition

2007-03-30T22:13:51Z

Erin-Walker: /* Findings */

== Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition ==
''Erin Walker, Bruce M. McLaren, Nikol Rummel, and Ken Koedinger''

=== Summary Tables ===
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''PIs''' || Bruce McLaren, Nikol Rummel
|-
| '''Other Contributers''' ||
* Graduate Student: Erin Walker
* Staff: Jonathan Steinhart
|}
 
'' Pre Study 1''
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''Study Start Date''' || 05/06
|-
| '''Study End Date''' || 06/06
|-
| '''LearnLab Site''' || CWCTC
|-
| '''LearnLab Course''' || Algebra
|-
| '''Number of Students''' || ''N'' = 14
|-
| '''Average # of hours per participant''' || 3 hrs.
|}
 

'' Pre Study 2''
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''Study Start Date''' || 11/06
|-
| '''Study End Date''' || 11/06
|-
| '''LearnLab Site''' || CWCTC
|-
| '''LearnLab Course''' || Algebra
|-
| '''Number of Students''' || ''N'' = 20
|-
| '''Average # of hours per participant''' || 3 hrs.
|}
 

'' Full Study''
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''Study Start Date''' || 04/07
|-
| '''Study End Date''' || 04/07
|-
| '''LearnLab Site''' || CWCTC
|-
| '''LearnLab Course''' || Algebra
|-
| '''Number of Students''' || ''N'' = 70
|-
| '''Average # of hours per participant''' || 3 hrs.
|}
 

=== Abstract ===
In this project, the Algebra I Cognitive Tutor is extended to a [[collaborative learning environment]]: Instead of the computer tutoring a student, students tutor each other, and the [[cognitive tutor]] provides collaborative support. As research has shown, collaborative problem solving and learning has the potential to increase elaboration on the learning content. However, students are not always able to effectively meet the challenges of a collaborative setting. To ensure that students capitalize on collaborative problem solving with the Tutor, a peer tutoring [[collaboration scripts|collaboration script]] was developed that guides their interaction and prompts fruitful [[collaboration]].

In the baseline version of the script (without cognitive tutor support), students in the same class are seated at different computers and take turs tutoring each other. First, they prepare to tutor by solving problems using the regular cognitive tutor. Then, they tutor each other by marking each other right and wrong, monitoring their partner's skills, and give each other hints and feedback. After preparing and tutoring, they reflect on the problems and their interaction, engaging in [[metacognition|metacognitive activities]]. We have expanded the script to include intelligent cognitive support of the peer tutoring, and hope to ultimately expand it to include intelligent collaborative support.

To assess the effectiveness of the script on [[robust learning]], we are conducting a series of [[in vivo experiment]]s in the Algebra LearnLab. In an initial, small scale study (pre study) that served to establish basic effects and to test the procedure in a classroom setting, we compared peer tutoring with reflection elements to peer tutoring without reflection elements. In a second study that served to explore the effects of cognitive tutoring of peer tutoring, we compared peer tutoring plus cognitive tutoring to peer tutoring. In the full study, we will compare these collaborative conditions to an individual condition to assess the effect of the collaborative Tutor extension to regular Tutor use.

=== Background and Significance ===

In our project, we combined two different [[instructional method]]s both of which have been shown to improve students’ learning in mathematics: Learning with intelligent tutoring systems (Koedinger, Anderson, Hadley, & Mark, 1997) and collaborative problem solving (Berg, 1993). The [[Cognitive tutor|Cognitive Tutor]] Algebra that was used in our study is a tutor for mathematics instruction at the high school level. Students learn with the Tutor during part of the regular classroom sessions. The Tutor's main features are immediate error feedback, the possibility to ask for a hint when encountering impasses, and knowledge tracing, i.e. the Tutor creates and updates a model of the student’s knowledge and selects new problems tailored to the student’s knowledge level. Although several studies have proven its effectiveness, students do not always benefit from learning with the Tutor. First, because the Tutor places emphasis on learning [[procedural|procedural problem solving skills]], yet a deep understanding of underlying mathematical [[conceptual knowledge|concepts]] is not necessarily achieved (Anderson, Corbett, Koedinger, & Pelletier, 1995). Second, students do not always make good use of the learning opportunities provided by the Cognitive Tutor (e.g. [[help abuse]], see Aleven, McLaren, Roll, & Koedinger, 2004; [[game the system|gaming the system]], Baker, Corbett, & Koedinger, 2004). So far, the Cognitive Tutor has been used in an individual learning setting only. However, as research on collaborative learning has shown, collaboration can yield elaboration of learning content (Teasley, 1995), thus this could be a promising approach to reduce the Tutor’s shortcomings. On the other hand, students are not always able to effectively meet the challenges of a collaborative setting (Rummel & Spada, 2005). [[Collaboration scripts]] have proven effective in helping people meet the challenges encountered when learning or working collaboratively (Kollar, Fischer, & Hesse, in press).

Combining collaborative activities with intelligent tutoring might combine the benefits of both approaches. Collaborative interaction could augment the effects of an existing ITS by adding deeper interaction, and intelligent tutoring support could improve the quality of student collaboration by providing guidance to students as they attempt to follow a collaboration script. One way to incorporate collaboration into an ITS is to place the intelligent tutoring system in the role of one of the collaborators and have a student interact with this agent. For example, Biswas, Schwartz, Leelawong, Vye, and the TAG-V (2005) have the student collaborate with an agent (“Betty”) in a peer tutoring scenario. The student teaches Betty about ecosystems with the help of a second mentoring agent (“Mr. Davis”), and it has been shown that a student who takes the role of a teacher in this system benefits more than a student who takes the role of a learner in a parallel traditional tutoring system. However, this system does not yet support natural language interaction between the agents involved, and therefore does not take full advantage of important collaborative interaction mechanisms that can increase learning. Another way to combine collaboration and tutoring is to use the ITS to structure and tutor the student collaboration, by connecting collaborative tools with ITS components (see Harrer, McLaren, Walker, Bollen, & Sewall, 2006), and developing a model of good and ineffective collaboration in order to provide on-line hints and feedback (see Soller, 2001). Most research into computer tutoring of the collaboration between two students is still in a preliminary stage.

We integrate collaborative learning with an ITS using a peer tutoring framework. Similar to Biswas et al. (2005), we put the student in the role of the tutor, and intend to have the cognitive elements of the tutoring supported by another agent. However, unlike Biswas et al., (2005), the teachable agent is a real student. Therefore, we have intelligent tutoring components monitoring and providing feedback on student collaboration in addition to student cognition, like Harrer et al. (2006). Our ultimate goal is to allow students to tutor each other through the interface of an ITS, supported by both cognitive and collaborative tutoring.

=== Glossary ===
See [[:Category:Peer Tutoring|Peer Tutoring Glossary]]

=== Research question ===
Does [[collaboration]] – and in particular [[collaboration scripts|scripted collaboration]] – improve students’ [[robust learning]] in the domain of algebra?

Do [[instructional method|instructional activities]] that involve three agents (a student, a peer tutor, and a computer tutor) increase [[robust learning]] compared to instructional activities that involve two agents (student and peer tutor, student and computer tutor)?

=== Independent variables ===
We vary the [[agent]]s involved in the interaction using the [[instructional method|mode of instruction]]. There are four conditions:
* Individual condition: Students solved problems with the Algebra I Cognitive Tutor in the regular fashion (i.e. computer program as additional [[agent]]).
* Peer tutoring condition: Students tutored each other using the cognitive tutor interface (i.e., another peer as additional [[agent]]). They went through two phases:
** a preparation phase, where students prepared to tutor while solving problems with the Algebra I Cognitive Tutor in the regular fashion
** a collaboration phase, where students take turns tutoring each other. Peer tutors can mark answers right or wrong, monitor their partner's skills, and give hints and feedback
* Peer tutoring plus reflection: As in the peer tutoring condition, students go through a preparation phase and then tutor each other in the collaboration phase. However, students also go through a series of [[metacognition|metacognitive]] self-reflection activities to encourage them to engage in [[elaborative interaction]]
* Peer tutoring plus cognitive tutoring: Students participate in the peer tutoring plus reflection condition, but during the collaboration phase, peer tutors receive cognitive tutor hints and feedback (i.e., a peer and computer program as additional [[agent]]s). This feedback contains two components:
** a collaborative prompt that encourages students to interact with each other
** a cognitive hint targeted at the particular mistake the peer tutee is making

=== Hypothesis ===
1. We expect the collaborative conditions to outperform the individual conditions in measures of robust learning of the algebra content.

2. We epect the peer tutoring plus cognitive tutoring condition to outperform the peer tutoring alone conditions in measures of robust learning of the algebra content

=== Dependent variables ===
* ''Near [[transfer]], immediate'': During training, student progress on training problems are be analyzed.
* ''Near [[transfer]], [[long-term retention|retention]]'': Students a are given a posttest immediately after the study on isomorphic problems
* ''Far [[transfer]]'': This paper and pencil test assessed students' understanding of the main mathematical concepts from the learning phase. The [[conceptual tasks]] students had to solve tapped the same knowledge components as the problems in instruction; however, the problems where non-isomorphic to those in the instruction, thus demanded students to flexibly apply their knowledge to problems with a new format.
* ''Acceleration of future learning'': Student learning on future equation solving units will be measured.

To compare [[collaboration skill]]s of students, we will be conducting an analysis of student dialogues durng the learning phase.

=== Explanation ===
This study is part of the [[Interactive_Communication|Interactive Communication cluster]], and its hypothesis is a specialization of the IC cluster’s central hypothesis. We assume that the instruction is in or above the student’s ZPD. Here is what we expect to happen in the three conditions:

* In the peer tutoring condition, the peer tutor’s help should increase learning compared to a student learning with the cognitive tutor. However, the student may make too many errors and/or require too much communication with the second agent. The peer tutor may not be able to provide the student with the help he/she needs. Learning will not be optimized.
* In the cognitive+peer tutoring condition, the peer tutor’s help should provide the additional benefits of interactive communication, and the cognitive tutor’s help should ensure that both the student and the peer tutor get the problem solving assistance they need. The support of the cognitive tutor and interaction with the student will also increase the peer tutor’s learning.
The target path will have the students learning-by-doing and learning-by-teaching, with the help of the computer agent so students do not make too many errors or require too much communication.

=== Findings ===
Pre-study 1:

Our first study compared the peer tutoring to peer tutoring plus reflection conditions. Using pretest and posttest scores, we conducted a two-way (condition x test-time) repeated-measure ANOVA, with test-time as the repeated measure. Posttest scores were significantly higher than pretest scores in both the tutoring and the tutoring+reflection condition (F (1,12) = 15.25, p < .002, η² = 0.56), but there were no significant differences between conditions, and no interaction (see Table 2). To further examine what occurred during the collaboration phase we turned to log data and notes from classroom observation. During peer tutoring, students appeared engaged, and did exhibit many of the positive collaborative behaviors that we were attempting to encourage with our script and that have been shown to correlate with knowledge construction and self-reflection. However, we observed that peer tutors struggled to provide tutees with answers, and did not connect the preparation that they had done with the collaboration phase. For instance, they often did not consult their answer printouts when they did not know the next problem step and thus had to rely on teacher assistance to solve a problem. As a result, tutees skipped problems without completing them correctly. This undesirable behavior differed between the two conditions. Students in the tutoring condition attempted more problems than students in the tutoring+reflection condition, and appeared to complete more problems as well. The average number of problems completed by dyads in the tutoring+reflection condition was low; students in this group took an average of 11 minutes to complete a single problem, compared to a 6 minute average in the tutoring condition. Students in the tutoring condition tended to skip problems they could not solve, completing less than 60% of the problems they attempted. Immediately before skipping a problem, students would generally state their inability to solve it, “I don’t know how to do this one,“ or their lack of motivation, “Just do something and I’ll agree or something.” If students skip problems, they may not learn how to solve difficult problems. However, if they do not complete many problems, they may not be sufficiently exposed to all the skills involved in the unit, and will be given fewer opportunities to master them.

Results for Pre-study 2 and Study 1 coming soon.

=== Annotated bibliography ===
Walker, E., Rummel, N., McLaren, B. M. & Koedinger, K. R. (submitted). The Student Becomes the Master: Integrating Peer Tutoring with Cognitive Tutoring. Submitted to the conference on Computer Supported Collaborative Learning (CSCL-07). Rutgers University, July 16-21, 2007.

Walker, E., Koedinger, K., McLaren, B. M., & Rummel, N. (2006). Cognitive tutors as research platforms: Extending an established tutoring system for collaborative and metacognitive experimentation. ''Lecture Notes in Computer Science, Volume 4053/2006. Proceedings of the 8th International Conference on Intelligent Tutoring Systems'' (pp. 207-216). Berlin: Springer

Walker, E. (2005). Mutual peer tutoring: A collaborative addition to the Algebra-1 Cognitive Tutor. Paper presented at the 12th International Conference on Artificial Intelligence and Education (AIED-05, Young Researchers Track), July, 2005, Amsterdam, the Netherlands.

=== References ===

Biswas, G., Schwartz, D. L., Leelawong, K., Vye, N., & TAG-V. (2005). Learning by teaching: A new agent paradigm for educational software. Applied Artificial Intelligence, 19, 363–392.

Carmien, S., Kollar, I., Fischer, G. & Fischer, F. (2006). The interplay of internal and external scripts. In F. Fischer, I. Kollar, H. Mandl &, J. Haake, Scripting computer-supported communication of knowledge. Cognitive, computational, and educational perspectives (pp. 289-311). New York: Springer.

Fantuzzo, J. W., Riggio, R. E., Connelly, S., & Dimeff, L. A. (1992). Effects of reciprocal peer tutoring on mathematics and school adjustment: A componential analysis. Journal of Educational Psychology, 84(3), 331-339.

Fuchs, L.S., Fuchs, D., Prentice, K., Burch, M., Hamlett, C.L., Owen, R., & Schroeter, K. (2003). Enhancing third-grade students' mathematical problem solving with self-regulated learning strategies. Journal of Educational Psychology, 95(2), 306-315.

Harrer, A., McLaren, B. M., Walker, E., Bollen, L., and Sewall, J. (2006). Creating Cognitive Tutors for Collaborative Learning: Steps Toward Realization. User Modeling and User-Adapted Interaction: The Journal of Personalization Research (UMUAI) (2006) 16: 175-209.

Johnson, D. W. and Johnson, R. T. (1990). Cooperative learning and achievement. In S. Sharan (Ed.), Cooperative learning: Theory and research (pp. 23-37). New York: Praeger.

King, A., Staffieri, A., & Adelgais, A. (1998). Mutual peer tutoring: Effects of structuring tutorial interaction to scaffold peer learning. Journal of Educational Psychology, 90, 134-152.

Koedinger, K. R., Anderson, J. R., Hadley, W. H., & Mark, M. A. (1997). Intelligent tutoring goes to school in the big city. International Journal of Artificial Intelligence in Education, 8, 30-43.

Kollar, I., Fischer, F., & Hesse, F. W. (in press). Collaboration scripts - a conceptual analysis. Educational Psychology Review.

Ploetzner, R., Dillenbourg, P., Preier, M.,&Traum, D. (1999). Learning by explaining to oneself and to others. In P. Dillenbourg (Ed.), Collaborative learning. Cognitive and computational approaches (pp. 103–121). Amsterdam: Pergamon.

Ritter, S., Blessing, S. B., & Hadley, W. S. (2002). SBIR Phase I Final Report 2002. Department of Education. Department of Education RFP ED: 84-305S.

Rummel, N., & Spada, H. (2005). Learning to collaborate: An instructional approach to promoting collaborative problem-solving in computer-mediated settings. Journal of the Learning Sciences, 14(2), 201--241.

Soller, A.L. (2001). Supporting Social Interaction in an Intelligent Collaborative Learning System. International Journal of Artificial Intelligence in Education, 12, 40-62.

Teasley, S. D. (1995). The role of talk in children's peer collaborations. Developmental Psychology, 31(2), 207-220.

Walker, E., Koedinger, K. R., McLaren, B. M. and Rummel, N. Cognitive Tutors as Research Platforms: Extending an Established Tutoring System for Collaborative and Metacognitive Experimentation. In the Proceedings of the 8th International Conference on Intelligent Tutoring Systems, Jhongli, Taiwan, June 26-30, 2006.

Webb, N.M., Troper, J.D., & Fall, R. (1995). Constructive activity and learning in collaborative small groups. Journal of Educational Psychology, 87, 406-423.

Walker A Peer Tutoring Addition

2007-03-30T22:10:39Z

Erin-Walker: /* Dependent variables */

Walker A Peer Tutoring Addition

2007-03-30T21:36:46Z

Erin-Walker: /* Hypothesis */

Walker A Peer Tutoring Addition

2007-03-30T21:36:13Z

Erin-Walker: /* Hypothesis */

Walker A Peer Tutoring Addition

2007-03-30T21:31:58Z

Erin-Walker: /* Independent variables */

Walker A Peer Tutoring Addition

2007-03-30T21:31:27Z

Erin-Walker: /* Independent variables */

Walker A Peer Tutoring Addition

2007-03-30T21:30:58Z

Erin-Walker: /* Independent variables */

Walker A Peer Tutoring Addition

2007-03-30T21:30:16Z

Erin-Walker: /* Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition */

Walker A Peer Tutoring Addition

2007-03-30T21:29:06Z

Erin-Walker: /* Independent variables */

Walker A Peer Tutoring Addition

2007-03-30T21:28:31Z

Erin-Walker: /* Independent variables */

Walker A Peer Tutoring Addition

2007-03-30T21:27:22Z

Erin-Walker: /* Independent variables */

Walker A Peer Tutoring Addition

2007-03-30T21:04:42Z

Erin-Walker: /* Research question */