Adaptive Assistance for Peer Tutoring (Walker, Rummer, Koedinger)
Contents
Collaborative Extensions to the Cognitive Tutor Algebra: Adaptive Assistance for Peer Tutoring
Erin Walker, Nikol Rummel, and Ken Koedinger
Summary Tables
| PI | Erin Walker |
| Co-PIs | Nikol Rummel, Ken Koedinger |
= Abstract
Adaptive collaborative learning support, where an intelligent system assesses student collaboration as it occurs and provides assistance when necessary, is a promising area of research. While fixed forms of support such as scripting student interaction have had a positive effect on collaboration quality, they can overconstrain the interaction for some students and provide too little help for others. Using intelligent tutoring technology to support collaboration might be more effective, but little is known about how to build these adaptive systems for collaboration and what effects they might have. We explore this area of research by augmenting an existing intelligent tutoring system with a peer tutoring activity and providing automated adaptive support to the activity.
This project has focused on how to improve the construction of adaptive collaboration systems with respect to their suitability for classroom deployment and the breadth of the models they employ. Most currently implemented systems are prototypes which are limited both in the scope of interaction that they support and in their use by students. In a previous PSLC project, “Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition, we explored the advantages of refactoring an existing intelligent tutoring system in order to transform it into a platform for collaborative research, such that interface and tutoring components can be added and removed in order to create different research conditions. We next demonstrated how individual intelligent tutoring models could be used as input to collaboration models in order to better assess peer tutoring behaviors. We propose to further extend this work by examining how individual models of student domain skills can be used as input to interaction models, and how collaborative skills can be represented. A related area of research is the potential of adaptive support for improving student interaction. The majority of the adaptive collaborative learning systems that have been developed have not been evaluated, and thus it is still unclear what influence adaptive support has compared to other form of support. Our first step in this area was to develop adaptive domain support for the peer tutor, and compare it to a condition where the peer tutor is simply given problem solutions. While both types of support had advantages and disadvantages, it was clear peer tutors needed assistance that targeted collaboration skills in addition to domain knowledge. The next iteration of the system added adaptive interaction support to the adaptive domain support, and currently we am comparing the combined assistance to a fixed condition in a classroom study. Our first proposed step will be to analyze the study data in order to identify the broad impact both types of support have on the quality of student interaction. We will then investigate in more detail the potential role adaptive feedback could play in assisting student interaction by: 1) using HCI design methodologies to examine how students perceive and react to different features of support, and 2) empirically evaluating how the directness of different support presentations influences student behavior. As an outcome of this research, we expect to add to understanding of the mechanisms by which adaptive support has an impact on student interaction, and how the support should be provided.
Background and Significance
Peer Tutoring: Learning by Teaching
Incorporating peer tutoring into the CTA might be a way to encourage deep learning. Roscoe and Chi conclude that peer tutors benefit due to knowledge-building, where they reflect on their current knowledge and use it as a basis for constructing new knowledge (Roscoe & Chi, 2007). Because these positive effects are independent of tutor domain ability, researchers implement reciprocal peer tutoring programs, where students of similar abilities take turns tutoring each other. This type of peer tutoring has been shown to increase academic achievement and positive attitudes in long-term classroom interventions (Fantuzzo, Riggio, Connely, & Dimeff, 1989). Biswas et al. (2005) described three properties of peer tutoring related to tutor learning: tutors are accountable for their tutee’s knowledge, they reflect on tutee actions, and they engage in asking questions and giving explanations. Tutee learning is maximized at times when the tutee reaches an impasse, is prompted to find and explain the correct step, and is given an explanation if they fail to do so (VanLehn et al., 2003).
Peer tutors rarely exhibit knowledge-building behaviors spontaneously (Roscoe & Chi, 2007), and thus successful interventions provide them with assistance in order to achieve better learning outcomes for them and their tutees. This assistance can target tutoring behaviors through training, providing positive examples, or structuring the tutoring activities. For example, training students to give conceptual explanations had a significantly positive effect on learning (Fuchs et al., 1997). It is just as critical for assistance to target domain expertise of the peer tutors, in order to ensure that they have sufficient knowledge about a problem to help their partner solve it. Otherwise, there may be cognitive consequences (tutees cannot correctly solve problems) and affective consequences (students feel that they are poor tutors and become discouraged; Medway & Baron, 1997). Domain assistance can take the form of preparation on the problems and scaffolding during tutoring (e.g., Fantuzzo, Riggio, Connely, & Dimeff, 1989). Although assistance for peer tutoring has generally been fixed, providing adaptive support may be a promising approach.
Adaptive Collaborative Learning Systems
In order to benefit from collaboration students must interact in productive ways, and collaborative activities can be structured (scripted) to encourage these behaviors (e.g., Rummel & Spada, 2007). However, fixed scripts implemented in a one-size-fits-all fashion may be too restrictive for some students and place a high cognitive demand on others (Rummel & Spada, 2007; Dillenbourg, 2002). An adaptive system would be able to monitor student behaviors and provide support only when needed. Preliminary results suggest that adaptive support is indeed beneficial: Adaptive prompting realized in a Wizard of Oz fashion has been shown to have a positive effect on interaction and learning compared to an unscripted condition (Gweon, Rose, Carey, & Zaiss, 2006). An effective way to deliver this support would be to use an adaptive collaborative learning system, where feedback on collaboration is delivered by an intelligent agent.
Work on adaptive collaborative learning systems is still at an early stage. One approach is to use machine learning to detect problematic elements of student interaction in real-time and trigger helpful prompts. Although implementations have lead to significant learning gains, the adaptive feedback appears to be disruptive to dyadic interaction (Kumar et al., 2007). Another promising approach has explored using an intelligent agent as one of the collaborators; students teach the agent about ecosystems with the help of a mentoring agent (Biswas et al. 2005). However, the agents do not interact with the students in natural language, one of the primary benefits of collaboration.
With respect to peer tutoring, intelligent tutoring technology could be applied either to supporting tutor behaviors or domain knowledge of peer tutors. As it is very difficult to build an intelligent tutor for collaborative processes, we decided to develop a general script for the peer tutoring interaction and then focus on providing adaptive domain assistance to peer tutors by leveraging the existing domain models of the CTA. A condition where students tutor each other with adaptive domain support provided to the peer tutor is likely to be better than a condition where the peer tutor merely has access to an answer key, because the support would be tailored to each individual tutor’s needs. It is also likely to be better than a condition where students use the CTA individually, because the students in the collaborative condition would be able to interact deeply about the domain material.
Glossary
Research Question
Can individual problem-solving models improve the effectiveness of adaptive collaborative learning support by providing more problem-solving context for models of collaboration? Do they make it easier to construct adaptive collaborative learning support systems?
What are the differential effects of adaptive and fixed support on student collaborative process during a peer tutoring activity, the acquisition of help-giving skills, and the resulting robust learning outcomes? How do the effects vary with different types of support?
Independent variables
We vary the agents involved in the interaction using the mode of instruction. For example, students may interact individually with a cognitive tutor, may interact with each other using a collaboration script, or may interact with each other with the help of a cognitive tutor.
In a series of studies, we further examine the type of assistance provided to the students. They may receive assistance on how to solve the problem, what good collaboration is, and how to collaborate well.
Finally, we vary within subjects whether students take on the tutor or tutee role.
Currently, we vary whether students collaborate or work individually, and the adaptivity of the support they receive:
Figure 1. Individual learning in the Cognitive Tutor Algebra.
Figure 2. Peer tutoring in the Cognitive Tutor Algebra. Peer tutor's interface.
Figure 3. Peer tutoring in the Cognitive Tutor Algebra. Adaptive domain support received by the peer tutor.
Figure 4. Peer tutoring in the Cognitive Tutor Algebra. Adaptive interaction support received by the peer tutor.
Hypotheses
1. Students that show high tutoring competence behaviors will show more domain learning than students that show low tutoring competence behaviors
2. Either domain or collaboration assistance alone is better than no assistance at promoting good peer tutoring behaviors. Providing peer tutors with both domain and collaboration assistance will lead them to show better peer tutoring behaviors than with only domain or collaboration assistance.
4. An increase in positive peer tutoring behaviors due to fixed assistance will lead to an increase in peer tutor robust domain learning
5. An increase in positive peer tutoring behaviors due to fixed assistance will lead to an increase in peer tutee robust domain learning
6. Adaptive assistance is more effective than fixed assistance at improving peer tutoring behaviors, which promotes robust domain learning of the peer tutor and peer tutee.
7. Students who receive fixed assistance will learn more from being tutors, while students who receive adaptive assistance will learn equally from both roles.
Dependent variables
- Normal post-test: Students are given a brief post-test immediately after each study day on isomorphic problems
- Far transfer: This paper and pencil test assessed students' understanding of the main mathematical concepts from the learning phase. The transfer items students had to solve tapped the same knowledge components as the problems in instruction, however, the problems where non-isomorphic to those in the instruction, thus demanded students to flexibly apply their knowledge to problems with a new format.
- Long-term retention: Students will be given a test a month after the immediate posttest.
- Accelerated future learning test: Student learning on future equation solving units will be measured.
To compare collaboration skills of students, we will be conducting an analysis of student dialogs during the learning phase.
To assess immediate effects of the instructional variations, we will analyze student progress on training problems as they work through the instruction.
Findings
We are in the process of conducting a classroom study in two schools with roughly 90 students in which we vary whether students receive adaptive or fixed assistance and whether they assume the tutor or tutee role first. Data collection is ongoing.
Out of this study, we expect to analyze in detail the effects of adaptive domain and interaction support on student interaction, student acquisition of collaborative skills, and domain learning.
Annotated bibliography
- Walker, E., Rummel, N., & Koedinger, K. R. Integrating collaboration and cognitive tutoring data in evaluation of a reciprocal peer tutoring environment. Submitted to Research and Practice in Technology Enhanced Learning. Accepted with minor revisions.
- Walker, E., Rummel, N., & Koedinger, K. R. CTRL: A Research Architecture for Providing Adaptive Collaborative Learning Support. Submitted to User Modeling
and User-Adapted Interaction. Accepted with minor revisions.
- Walker, E., Rummel, N., and Koedinger, K. R. To Tutor the Tutor: Adaptive Domain Support for Peer Tutoring. To appear at the 9th International Conference on Intelligent Tutoring Systems. 2008.
- Walker, E., McLaren, B. M., Rummel, N., and Koedinger, K. R. Who Says Three's a Crowd? Using a Cognitive Tutor to Support Peer Tutoring. 13th International Conference on Artificial Intelligence and Education. 2007.
- Walker, E., Rummel, N., McLaren, B. M. & Koedinger, K. R. The Student Becomes the Master: Integrating Peer Tutoring with Cognitive Tutoring. Short paper at the Conference on Computer Supported Collaborative Learning (CSCL-07). Rutgers University, July 16-21, 2007.
- Walker, E., Koedinger, K., McLaren, B. M., & Rummel, N. (2006). Cognitive tutors as research platforms: Extending an established tutoring system for collaborative and metacognitive experimentation. Lecture Notes in Computer Science, Volume 4053/2006. Proceedings of the 8th International Conference on Intelligent Tutoring Systems (pp. 207-216). Berlin: Springer
- Walker, E. (2005). Mutual peer tutoring: A collaborative addition to the Algebra-1 Cognitive Tutor. Paper presented at the 12th International Conference on Artificial Intelligence and Education (AIED-05, Young Researchers Track), July, 2005, Amsterdam, the Netherlands.
References
- Roscoe, R. D. & Chi, M. Understanding tutor learning: Knowledge-building and knowledge-telling in peer tutors’ explanations and questions. Review of Educational Research 77(4), 534-574 (2007)
- Fantuzzo, J. W., Riggio, R. E., Connelly, S., & Dimeff, L. A. Effects of reciprocal peer tutoring on academic achievement and psychological adjustment: A component analysis. Journal of Educational Psychology 81(2), 173-177 (1989)
- Biswas, G., Schwartz, D. L., Leelawong, K., Vye, N., & TAG-V. Learning by teaching: A new agent paradigm for educational software. Applied Artificial Intelligence 19, 363–392 (2005)
- VanLehn, K., Siler, S., Murray, C., Yamauchi, T., & Baggett, W. Why do only some events cause learning during human tutoring? Cognition and Instruction 21(3), 209-249 (2003)
- Fuchs, L., Fuchs, D., Hamlett, C., Phillips, N., Karns, K., & Dutka, S. Enhancing students’ helping behaviour during peer-mediated instruction with conceptual mathematical explanations. The Elementary School Journal 97(3), 223-249 (1997)
- Medway, F. & Baron, R. Locus of control and tutors’ instructional style. Contemporary Educational Psychology, 2, 298-310 (1997).
- Rummel, N. & Spada, H. Can people learn computer-mediated collaboration by following a script? In F. Fischer, I. Kollar, H. Mandl &, J. Haake, Scripting computer-supported communication of knowledge. Cognitive, computational, and educational perspectives (pp. 47-63). New York: Springer. (2007)
- Dillenbourg, P. Over-scripting CSCL: The risks of blending collaborative learning with instructional design. In P. A. Kirschner (Ed.), Three worlds of CSCL. Can we support CSCL (pp. 61-91). Heerlen: Open Universiteit Nederland. (2002)
- Gweon, G., Rosé, C., Carey, R. & Zaiss, Z. Providing Support for Adaptive Scripting in an On-Line Collaborative Learning Environment. Proc. of CHI 2006, pp. 251-260. (2006)
- Kumar, R., Rosé, C. P., Wang, Y. C., Joshi, M., Robinson, A. Tutorial dialogue as adaptive collaborative learning support. Proceedings of the 13th International Conference on Artificial Intelligence in Education (AIED 2007), Amsterdam: IOSPress. (2007)
Connections
This study is an extension of the PSLC project "Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition."
Like this study, Rummel Scripted Collaborative Problem Solving adds scripted collaborative problem solving to the Cognitive Tutor Algebra. The studies differ in the way collaboration is integrated in the Tutor. First, in the Rummel et al. study, both students first prepare one subtasks of a problem to mutually solve the complex story problem later on. Thus, although the students are experts for different parts of the problem, they have a comparable knowledge level during collaboration. In contrast, in this study, one student prepares to teach his partner. Then, they change roles. Thus, during collaboration, their knowledge level differs. Second, in the Rummel et al. study, collaboration was face to face, whereas this study used a chat tool for interaction.
Similar to the adaptive script component of the Collaborative Problem-Solving Script, the Help Tutor project aims at improving students' help-seeking behavior and at reducing students' tendency to game the system.
Furthermore, both studies contain instructions to teach metacognition. The metacognitive component in our study instructs students to monitor their interaction in order to improve it in subsequent collaborations; the Help Tutor project asks students to evaluate their need for help in order to improve their help-seeking behavior when learning on the Tutor.
Both in this study and in the Reflective Dialogue study from Katz, students are asked to engage in reflection following each problem-solving. In this study, the reflection concentrates on the collaborative skills, while in Katz' study, the reflection concentrates on students' domain knowledge of the main principles applied in the problem.
Furthermore, both our study and the Help Lite (Aleven, Roll) aim at improving conceptual knowledge.
Future plans
Our future plans for June 2009 - September 2009:
- Analyze the study data
