Adaptive Assistance for Peer Tutoring (Walker, Rummer, Koedinger)

From LearnLab
Revision as of 19:44, 14 April 2008 by Erin-Walker (talk | contribs) (Hypotheses)
Jump to: navigation, search

Collaborative Extensions to the Cognitive Tutor Algebra: Adaptive Assistance for Peer Tutoring

Erin Walker, Nikol Rummel, and Ken Koedinger

Summary Tables

PI Erin Walker
Co-PIs Nikol Rummel, Ken Koedinger


Abstract

Our research goal is to integrate a peer tutoring script within the context of the Cognitive Tutor Algebra (CTA) allowing students to tutor each other through the interface of an intelligent tutoring system (ITS) that provides both domain support and collaborative tutoring. In the PSLC project, “Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition,” we have added peer tutoring to the Cognitive Tutor Algebra and developed adaptive domain support for the peer tutor. We propose to continue this work by developing and evaluating collaborative assistance for the peer tutor within this context. This assistance will target both the skills required to successfully tutor and the motivation for students to tutor. Once we have shown fixed collaborative assistance to be effective, we plan to implement it in an adaptive fashion, and compare the effects of adaptive and fixed assistance on collaborative skill acquisition and robust domain learning.

The integration of intelligent tutoring and collaborative learning allows us to investigate the differential effects of varying the type and adaptivity of assistance provided to collaborating peers on the acquisition of collaborative skills and on robust domain learning. Further, the development of a successful adaptive collaborative learning system would be a significant contribution to the ITS community.

Background and Significance

Peer Tutoring: Learning by Teaching

Incorporating peer tutoring into the CTA might be a way to encourage deep learning. Roscoe and Chi conclude that peer tutors benefit due to knowledge-building, where they reflect on their current knowledge and use it as a basis for constructing new knowledge (Roscoe & Chi, 2007). Because these positive effects are independent of tutor domain ability, researchers implement reciprocal peer tutoring programs, where students of similar abilities take turns tutoring each other. This type of peer tutoring has been shown to increase academic achievement and positive attitudes in long-term classroom interventions (Fantuzzo, Riggio, Connely, & Dimeff, 1989). Biswas et al. (2005) described three properties of peer tutoring related to tutor learning: tutors are accountable for their tutee’s knowledge, they reflect on tutee actions, and they engage in asking questions and giving explanations. Tutee learning is maximized at times when the tutee reaches an impasse, is prompted to find and explain the correct step, and is given an explanation if they fail to do so (VanLehn et al., 2003).

Peer tutors rarely exhibit knowledge-building behaviors spontaneously (Roscoe & Chi, 2007), and thus successful interventions provide them with assistance in order to achieve better learning outcomes for them and their tutees. This assistance can target tutoring behaviors through training, providing positive examples, or structuring the tutoring activities. For example, training students to give conceptual explanations had a significantly positive effect on learning (Fuchs et al., 1997). It is just as critical for assistance to target domain expertise of the peer tutors, in order to ensure that they have sufficient knowledge about a problem to help their partner solve it. Otherwise, there may be cognitive consequences (tutees cannot correctly solve problems) and affective consequences (students feel that they are poor tutors and become discouraged; Medway & Baron, 1997). Domain assistance can take the form of preparation on the problems and scaffolding during tutoring (e.g., Fantuzzo, Riggio, Connely, & Dimeff, 1989). Although assistance for peer tutoring has generally been fixed, providing adaptive support may be a promising approach.


Adaptive Collaborative Learning Systems

In order to benefit from collaboration students must interact in productive ways, and collaborative activities can be structured (scripted) to encourage these behaviors (e.g., Fischer, Kollar, Mandl, & Haake, 2007). However, fixed scripts implemented in a one-size-fits-all fashion may be too restrictive for some students and place a high cognitive demand on others (Fischer et al., 2007; Dillenbourg, 2002). An adaptive system would be able to monitor student behaviors and provide support only when needed. Preliminary results suggest that adaptive support is indeed beneficial: Adaptive prompting realized in a Wizard of Oz fashion has been shown to have a positive effect on interaction and learning compared to an unscripted condition (Gweon, Rose, Carey, & Zaiss, 2006). An effective way to deliver this support would be to use an adaptive collaborative learning system, where feedback on collaboration is delivered by an intelligent agent.

Work on adaptive collaborative learning systems is still at an early stage. One approach is to use machine learning to detect problematic elements of student interaction in real-time and trigger helpful prompts. Although implementations have lead to significant learning gains, the adaptive feedback appears to be disruptive to dyadic interaction (Kumar et al., 2007). Another promising approach has explored using an intelligent agent as one of the collaborators; students teach the agent about ecosystems with the help of a mentoring agent (Biswas et al. 2005). However, the agents do not interact with the students in natural language, one of the primary benefits of collaboration.

With respect to peer tutoring, intelligent tutoring technology could be applied either to supporting tutor behaviors or domain knowledge of peer tutors. As it is very difficult to build an intelligent tutor for collaborative processes, we decided to develop a general script for the peer tutoring interaction and then focus on providing adaptive domain assistance to peer tutors by leveraging the existing domain models of the CTA. A condition where students tutor each other with adaptive domain support provided to the peer tutor is likely to be better than a condition where the peer tutor merely has access to an answer key, because the support would be tailored to each individual tutor’s needs. It is also likely to be better than a condition where students use the CTA individually, because the students in the collaborative condition would be able to interact deeply about the domain material.

Glossary

See Peer Tutoring Glossary

Research Question

What are the differential effects of adaptive and fixed support on student collaborative process during a peer tutoring activity and the resulting robust learning outcomes?

How does an instructional method that provides metacognitive support and incentives for peer tutoring affect student collaborative process and robust learning outcomes?

Independent variables

We vary the agents involved in the interaction using the mode of instruction. For example, students may interact individually with a cognitive tutor, may interact with each other using a collaboration script, or may interact with each other with the help of a cognitive tutor.

We also plan to examine the type of assistance provided to the students. They may receive assistance to tutoring competence (either domain, metacognitive, or procedural) or motivational support.

Hypotheses

1. Students that show high tutoring competence behaviors will show more domain learning than students that show low tutoring competence behaviors

2. Student perceptions of their tutoring role and beliefs that they can fill that role are related to their use of effective tutoring behaviors.

3. Either competence or motivational assistance alone is better than no assistance at promoting good peer tutoring behaviors. Providing peer tutors with both competence and motivational assistance will lead them to show better peer tutoring behaviors than with only competence or motivational assistance.

4. An increase in positive peer tutoring behaviors due to fixed assistance will lead to an increase in peer tutor robust domain learning

5. An increase in positive peer tutoring behaviors due to fixed assistance will lead to an increase in peer tutee robust domain learning

6. Adaptive assistance is more effective than fixed assistance at improving peer tutoring behaviors, which promotes robust domain learning of the peer tutor and peer tutee.

Dependent variables

  • Normal post-test: Students are given a post-test immediately after the study on isomorphic problems
  • Far transfer: This paper and pencil test assessed students' understanding of the main mathematical concepts from the learning phase. The transfer items students had to solve tapped the same knowledge components as the problems in instruction, however, the problems where non-isomorphic to those in the instruction, thus demanded students to flexibly apply their knowledge to problems with a new format.
  • Long-term retention: Students will be given a test a month after the immediate posttest.
  • Accelerated future learning test: Student learning on future equation solving units will be measured.

To compare collaboration skills of students, we will be conducting an analysis of student dialogs during the learning phase.

To assess immediate effects of the instructional variations, we will analyze student progress on training problems as they work through the instruction.

Findings

We are still in the design phase of this project.

Annotated bibliography

  • Walker, E., Rummel, N., and Koedinger, K. R. To Tutor the Tutor: Adaptive Domain Support for Peer Tutoring. To appear at the 9th International Conference on Intelligent Tutoring Systems. 2008.
  • Walker, E., McLaren, B. M., Rummel, N., and Koedinger, K. R. Who Says Three's a Crowd? Using a Cognitive Tutor to Support Peer Tutoring. 13th International Conference on Artificial Intelligence and Education. 2007.
  • Walker, E., Rummel, N., McLaren, B. M. & Koedinger, K. R. The Student Becomes the Master: Integrating Peer Tutoring with Cognitive Tutoring. Short paper at the Conference on Computer Supported Collaborative Learning (CSCL-07). Rutgers University, July 16-21, 2007.
  • Walker, E., Koedinger, K., McLaren, B. M., & Rummel, N. (2006). Cognitive tutors as research platforms: Extending an established tutoring system for collaborative and metacognitive experimentation. Lecture Notes in Computer Science, Volume 4053/2006. Proceedings of the 8th International Conference on Intelligent Tutoring Systems (pp. 207-216). Berlin: Springer
  • Walker, E. (2005). Mutual peer tutoring: A collaborative addition to the Algebra-1 Cognitive Tutor. Paper presented at the 12th International Conference on Artificial Intelligence and Education (AIED-05, Young Researchers Track), July, 2005, Amsterdam, the Netherlands.

References

  • Roscoe, R. D. & Chi, M. Understanding tutor learning: Knowledge-building and knowledge-telling in peer tutors’ explanations and questions. Review of Educational Research 77(4), 534-574 (2007)
  • Fantuzzo, J. W., Riggio, R. E., Connelly, S., & Dimeff, L. A. Effects of reciprocal peer tutoring on academic achievement and psychological adjustment: A component analysis. Journal of Educational Psychology 81(2), 173-177 (1989)
  • Biswas, G., Schwartz, D. L., Leelawong, K., Vye, N., & TAG-V. Learning by teaching: A new agent paradigm for educational software. Applied Artificial Intelligence 19, 363–392 (2005)
  • VanLehn, K., Siler, S., Murray, C., Yamauchi, T., & Baggett, W. Why do only some events cause learning during human tutoring? Cognition and Instruction 21(3), 209-249 (2003)
  • Fuchs, L., Fuchs, D., Hamlett, C., Phillips, N., Karns, K., & Dutka, S. Enhancing students’ helping behaviour during peer-mediated instruction with conceptual mathematical explanations. The Elementary School Journal 97(3), 223-249 (1997)
  • Medway, F. & Baron, R. Locus of control and tutors’ instructional style. Contemporary Educational Psychology, 2, 298-310 (1997).

13. Rummel, N. & Spada, H. Can people learn computer-mediated collaboration by following a script? In * F. Fischer, I. Kollar, H. Mandl &, J. Haake, Scripting computer-supported communication of knowledge. Cognitive, computational, and educational perspectives (pp. 47-63). New York: Springer. (2007)

  • Dillenbourg, P. Over-scripting CSCL: The risks of blending collaborative learning with instructional design. In P. A. Kirschner (Ed.), Three worlds of CSCL. Can we support CSCL (pp. 61-91). Heerlen: Open Universiteit Nederland. (2002)
  • Gweon, G., Rosé, C., Carey, R. & Zaiss, Z. Providing Support for Adaptive Scripting in an On-Line Collaborative Learning Environment. Proc. of CHI 2006, pp. 251-260. (2006)
  • Kumar, R., Rosé, C. P., Wang, Y. C., Joshi, M., Robinson, A. Tutorial dialogue as adaptive collaborative learning support. Proceedings of the 13th International Conference on Artificial Intelligence in Education (AIED 2007), Amsterdam: IOSPress. (2007)

Connections

This study is an extension of the PSLC project "Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition."

Like this study, Rummel Scripted Collaborative Problem Solving adds scripted collaborative problem solving to the Cognitive Tutor Algebra. The studies differ in the way collaboration is integrated in the Tutor. First, in the Rummel et al. study, both students first prepare one subtasks of a problem to mutually solve the complex story problem later on. Thus, although the students are experts for different parts of the problem, they have a comparable knowledge level during collaboration. In contrast, in this study, one student prepares to teach his partner. Then, they change roles. Thus, during collaboration, their knowledge level differs. Second, in the Rummel et al. study, collaboration was face to face, whereas this study used a chat tool for interaction.

Similar to the adaptive script component of the Collaborative Problem-Solving Script, the Help Tutor project aims at improving students' help-seeking behavior and at reducing students' tendency to game the system.
Furthermore, both studies contain instructions to teach metacognition. The metacognitive component in our study instructs students to monitor their interaction in order to improve it in subsequent collaborations; the Help Tutor project asks students to evaluate their need for help in order to improve their help-seeking behavior when learning on the Tutor.

Both in this study and in the Reflective Dialogue study from Katz, students are asked to engage in reflection following each problem-solving. In this study, the reflection concentrates on the collaborative skills, while in Katz' study, the reflection concentrates on students' domain knowledge of the main principles applied in the problem.

Furthermore, both our study and the Help Lite (Aleven, Roll) aim at improving conceptual knowledge.


Future plans

Our future plans for June 2008 - December 2008:

  • Design & implement further assistance for peer tutoring
  • Run lab study evaluating the assistance