Features of Adaptive Assistance that Improve Peer Tutoring in Algebra (Walker, Rummel, Koedinger)

From LearnLab
Revision as of 20:30, 2 September 2011 by Mbett (talk | contribs) (Reverted edits by Elenilowery (Talk); changed back to last version by Erin-Walker)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Collaborative Extensions to the Cognitive Tutor Algebra: Adaptive Assistance for Peer Tutoring

Erin Walker, Nikol Rummel, and Ken Koedinger

Summary Tables

PI Erin Walker
Co-PIs Nikol Rummel, Ken Koedinger


Abstract

Adaptive collaborative learning support, where an intelligent system assesses student collaboration as it occurs and provides assistance when necessary, is a promising area of research. While fixed forms of support such as scripting student interaction have had a positive effect on collaboration quality, they can overconstrain the interaction for some students and provide too little help for others. Using intelligent tutoring technology to support collaboration might be more effective, but little is known about how to build these adaptive systems for collaboration and what effects they might have. We explore this area of research by augmenting an existing intelligent tutoring system with a peer tutoring activity and providing automated adaptive support to the activity.

This project has focused on how to improve the construction of adaptive collaboration systems with respect to their suitability for classroom deployment and the breadth of the models they employ. Most currently implemented systems are prototypes which are limited both in the scope of interaction that they support and in their use by students. In our first PSLC project, “Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition", we explored the advantages of refactoring an existing intelligent tutoring system in order to transform it into a platform for collaborative research, such that interface and tutoring components can be added and removed in order to create different research conditions. We next demonstrated how individual intelligent tutoring models could be used as input to collaboration models in order to better assess peer tutoring behaviors, in the PSLC project "Collaborative Extensions to the Cognitive Tutor Algebra: Adaptive Assistance for Peer Tutoring". In this project, we extend this work by examining how individual models of student domain skills can be used as input to interaction models.

A related area of research is the potential of adaptive support for improving student interaction. The majority of the adaptive collaborative learning systems that have been developed have not been evaluated, and thus it is still unclear what influence adaptive support has compared to other forms of support. Our first step in this area was to develop adaptive domain support for the peer tutor, and compare it to a condition where the peer tutor is simply given problem solutions. While both types of support had advantages and disadvantages, it was clear peer tutors needed assistance that targeted collaboration skills in addition to domain knowledge. The next iteration of the system added adaptive interaction support to the adaptive domain support, and we compared the combined assistance to a fixed condition in a classroom study. As part of this project, we analyzed the study data in order to identify the broad impact both types of support have on the quality of student interaction, finding that adaptive support improves the quality of student help over fixed support. We now propose to investigate in more detail the potential role adaptive feedback could play in assisting student interaction by: 1) using HCI design methodologies to examine how students perceive and react to different features of support, and 2) empirically evaluating whether adaptive support has a cognitive or motivational influence on students. As an outcome of this research, we expect to add to understanding of the mechanisms by which adaptive support has an impact on student interaction, and how the support should be provided.

Background & Significance

Peer Tutoring: Learning by Teaching

Incorporating peer tutoring into the CTA might be a way to encourage deep learning. Roscoe and Chi conclude that peer tutors benefit due to knowledge-building, where they reflect on their current knowledge and use it as a basis for constructing new knowledge (Roscoe & Chi, 2007). Because these positive effects are independent of tutor domain ability, researchers implement reciprocal peer tutoring programs, where students of similar abilities take turns tutoring each other. This type of peer tutoring has been shown to increase academic achievement and positive attitudes in long-term classroom interventions (Fantuzzo, Riggio, Connely, & Dimeff, 1989). Biswas et al. (2005) described three properties of peer tutoring related to tutor learning: tutors are accountable for their tutee’s knowledge, they reflect on tutee actions, and they engage in asking questions and giving explanations. Tutee learning is maximized at times when the tutee reaches an impasse, is prompted to find and explain the correct step, and is given an explanation if they fail to do so (VanLehn et al., 2003).

Peer tutors rarely exhibit knowledge-building behaviors spontaneously (Roscoe & Chi, 2007), and thus successful interventions provide them with assistance in order to achieve better learning outcomes for them and their tutees. This assistance can target tutoring behaviors through training, providing positive examples, or structuring the tutoring activities. For example, training students to give conceptual explanations had a significantly positive effect on learning (Fuchs et al., 1997). It is just as critical for assistance to target domain expertise of the peer tutors, in order to ensure that they have sufficient knowledge about a problem to help their partner solve it. Otherwise, there may be cognitive consequences (tutees cannot correctly solve problems) and affective consequences (students feel that they are poor tutors and become discouraged; Medway & Baron, 1997). Domain assistance can take the form of preparation on the problems and scaffolding during tutoring (e.g., Fantuzzo, Riggio, Connely, & Dimeff, 1989). Although assistance for peer tutoring has generally been fixed, providing adaptive support may be a promising approach.


Adaptive Collaborative Learning Systems

In order to benefit from collaboration students must interact in productive ways, and collaborative activities can be structured (scripted) to encourage these behaviors (e.g., Rummel & Spada, 2007). However, fixed scripts implemented in a one-size-fits-all fashion may be too restrictive for some students and place a high cognitive demand on others (Rummel & Spada, 2007; Dillenbourg, 2002). An adaptive system would be able to monitor student behaviors and provide support only when needed. Preliminary results suggest that adaptive support is indeed beneficial: Adaptive prompting realized in a Wizard of Oz fashion has been shown to have a positive effect on interaction and learning compared to an unscripted condition (Gweon, Rose, Carey, & Zaiss, 2006). An effective way to deliver this support would be to use an adaptive collaborative learning system, where feedback on collaboration is delivered by an intelligent agent.

Work on adaptive collaborative learning systems is still at an early stage. One approach is to use machine learning to detect problematic elements of student interaction in real-time and trigger helpful prompts. Although implementations have lead to significant learning gains, the adaptive feedback appears to be disruptive to dyadic interaction (Kumar et al., 2007). Another promising approach has explored using an intelligent agent as one of the collaborators; students teach the agent about ecosystems with the help of a mentoring agent (Biswas et al. 2005). However, the agents do not interact with the students in natural language, one of the primary benefits of collaboration. With respect to peer tutoring, intelligent tutoring technology could be applied either to supporting tutor behaviors or supporting the domain knowledge of peer tutors.

Glossary

See Peer Tutoring Glossary

Research Questions

Can individual problem-solving models improve the effectiveness of adaptive collaborative learning support by providing more problem-solving context for models of collaboration? Do they make it easier to construct adaptive collaborative learning support systems?

What are the differential effects of adaptive and fixed support on student collaborative process during a peer tutoring activity, the acquisition of help-giving skills, and the resulting robust learning outcomes? Does adaptive support improve student ability to collaborate, student motivation to collaborate, or both?


Independent Variables

1. Actual adaptivity of interaction support. We vary whether students are given support with highly relevant content at the moments they need it, or support with random content at moments when it is not needed. 2. Perceived adaptivity of interaction support. We vary whether students believe the support they are receiving is adaptively or randomly chosen. 3. Student role. We vary whether students take on the tutor or tutee role.




Peer tutoring in the Cognitive Tutor Algebra. Adaptive interaction support received by the peer tutor.
Walker adaptive interaction support.jpg


Hypotheses

1. Peer tutors that show effective tutoring behaviors will show more domain learning than students that show ineffective tutoring behaviors.

2. Peer tutees that receive good tutoring will show more domain learning than peer tutees that receive bad tutoring.

3. Peer tutors that believe the assistance that they are receiving is adaptive will improve the quality of their tutoring, because they will feel more accountable for their behaviors.

4. Peer tutors that receive adaptive assistance will improve the quality of their tutoring, because they will be able to more easily apply the assistance to their behaviors.


Dependent variables

  • Normal post-test: Students are given a brief post-test immediately after each study day on isomorphic problems
  • Far transfer: This paper and pencil test assessed students' understanding of the main mathematical concepts from the learning phase. The transfer items students had to solve tapped the same knowledge components as the problems in instruction, however, the problems where non-isomorphic to those in the instruction, thus demanded students to flexibly apply their knowledge to problems with a new format.
  • Collaboration posttest: Students collaborate without support in order to determine if they've improved their tutoring skills.

To compare collaboration skills of students, we will be conducting an analysis of student dialogs during the learning phase.

To assess immediate effects of the instructional variations, we will analyze student progress on training problems as they work through the instruction.


Findings

We are in the process of conducting a lab study with roughly 120 students. Out of this study, we expect to analyze in detail the effects of adaptive interaction support on student interaction, student acquisition of collaborative skills, and domain learning.

Annotated bibliography

  • Walker, E., Rummel, N., & Koedinger, K. R. Integrating collaboration and cognitive tutoring data in evaluation of a reciprocal peer tutoring environment. Research and Practice in Technology Enhanced Learning.
  • Walker, E., Rummel, N., & Koedinger, K. R. CTRL: A Research Architecture for Providing Adaptive Collaborative Learning Support. User Modeling and User-Adapted Interaction.
  • Walker, E., Rummel, N., and Koedinger, K. R. To Tutor the Tutor: Adaptive Domain Support for Peer Tutoring. To appear at the 9th International Conference on Intelligent Tutoring Systems. 2008.
  • Walker, E., McLaren, B. M., Rummel, N., and Koedinger, K. R. Who Says Three's a Crowd? Using a Cognitive Tutor to Support Peer Tutoring. 13th International Conference on Artificial Intelligence and Education. 2007.
  • Walker, E., Rummel, N., McLaren, B. M. & Koedinger, K. R. The Student Becomes the Master: Integrating Peer Tutoring with Cognitive Tutoring. Short paper at the Conference on Computer Supported Collaborative Learning (CSCL-07). Rutgers University, July 16-21, 2007.
  • Walker, E., Koedinger, K., McLaren, B. M., & Rummel, N. (2006). Cognitive tutors as research platforms: Extending an established tutoring system for collaborative and metacognitive experimentation. Lecture Notes in Computer Science, Volume 4053/2006. Proceedings of the 8th International Conference on Intelligent Tutoring Systems (pp. 207-216). Berlin: Springer
  • Walker, E. (2005). Mutual peer tutoring: A collaborative addition to the Algebra-1 Cognitive Tutor. Paper presented at the 12th International Conference on Artificial Intelligence and Education (AIED-05, Young Researchers Track), July, 2005, Amsterdam, the Netherlands.

References

  • Roscoe, R. D. & Chi, M. Understanding tutor learning: Knowledge-building and knowledge-telling in peer tutors’ explanations and questions. Review of Educational Research 77(4), 534-574 (2007)
  • Fantuzzo, J. W., Riggio, R. E., Connelly, S., & Dimeff, L. A. Effects of reciprocal peer tutoring on academic achievement and psychological adjustment: A component analysis. Journal of Educational Psychology 81(2), 173-177 (1989)
  • Biswas, G., Schwartz, D. L., Leelawong, K., Vye, N., & TAG-V. Learning by teaching: A new agent paradigm for educational software. Applied Artificial Intelligence 19, 363–392 (2005)
  • VanLehn, K., Siler, S., Murray, C., Yamauchi, T., & Baggett, W. Why do only some events cause learning during human tutoring? Cognition and Instruction 21(3), 209-249 (2003)
  • Fuchs, L., Fuchs, D., Hamlett, C., Phillips, N., Karns, K., & Dutka, S. Enhancing students’ helping behaviour during peer-mediated instruction with conceptual mathematical explanations. The Elementary School Journal 97(3), 223-249 (1997)
  • Medway, F. & Baron, R. Locus of control and tutors’ instructional style. Contemporary Educational Psychology, 2, 298-310 (1997).
  • Rummel, N. & Spada, H. Can people learn computer-mediated collaboration by following a script? In F. Fischer, I. Kollar, H. Mandl &, J. Haake, Scripting computer-supported communication of knowledge. Cognitive, computational, and educational perspectives (pp. 47-63). New York: Springer. (2007)
  • Dillenbourg, P. Over-scripting CSCL: The risks of blending collaborative learning with instructional design. In P. A. Kirschner (Ed.), Three worlds of CSCL. Can we support CSCL (pp. 61-91). Heerlen: Open Universiteit Nederland. (2002)
  • Gweon, G., Rosé, C., Carey, R. & Zaiss, Z. Providing Support for Adaptive Scripting in an On-Line Collaborative Learning Environment. Proc. of CHI 2006, pp. 251-260. (2006)
  • Kumar, R., Rosé, C. P., Wang, Y. C., Joshi, M., Robinson, A. Tutorial dialogue as adaptive collaborative learning support. Proceedings of the 13th International Conference on Artificial Intelligence in Education (AIED 2007), Amsterdam: IOSPress. (2007)


Connections

This study is an extension of the PSLC project "Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition" and "Collaborative Extensions to the Cognitive Tutor Algebra: Adaptive Assistance for Peer Tutoring."

Like this study, Rummel Scripted Collaborative Problem Solving adds scripted collaborative problem solving to the Cognitive Tutor Algebra. The studies differ in the way collaboration is integrated in the Tutor. First, in the Rummel et al. study, both students first prepare one subtasks of a problem to mutually solve the complex story problem later on. Thus, although the students are experts for different parts of the problem, they have a comparable knowledge level during collaboration. In contrast, in this study, one student prepares to teach his partner. Then, they change roles. Thus, during collaboration, their knowledge level differs. Second, in the Rummel et al. study, collaboration was face to face, whereas this study used a chat tool for interaction.

Similar to the adaptive script component of the Collaborative Problem-Solving Script, the Help Tutor project aims at improving students' help-seeking behavior and at reducing students' tendency to game the system.
Furthermore, both studies contain instructions to teach metacognition. The metacognitive component in our study instructs students to monitor their interaction in order to improve it in subsequent collaborations; the Help Tutor project asks students to evaluate their need for help in order to improve their help-seeking behavior when learning on the Tutor.

Both in this study and in the Reflective Dialogue study from Katz, students are asked to engage in reflection following each problem-solving. In this study, the reflection concentrates on the collaborative skills, while in Katz' study, the reflection concentrates on students' domain knowledge of the main principles applied in the problem.

Furthermore, both our study and the Help Lite (Aleven, Roll) aim at improving conceptual knowledge.

This project relates the more general thrust goals as follows. It is examining how features of assistance affect the three aspects of accountable talk: accountability to knowledge, accountability to rigorous thinking, and accountability to the learning community. Steps are being made toward the ambitious goal to operationalize and assess these aspects of accountable in real time as students interact and receive assistance in this computer-mediated environment. There is also a potential to code the three way dialog (student tutee, student tutor, and computer tutor) for transactivity. In particular, the student tutee's dialog moves have not yet been coded, but appear to have interesting elements, like asking for specific help or self-explaining, that may well connect to transactivity codes. Finally, there is a potential to analyze the computer tutor's reflective prompts for similarity with accountable talk moves and associated effectiveness. Some of the prompts were indeed inspired by accountable talk moves.