The Help Tutor Roll Aleven McLaren

From LearnLab
Revision as of 18:14, 19 September 2006 by Koedinger (talk | contribs) (Findings)
Jump to: navigation, search

Towards Tutoring Metacognition - The Case of Help Seeking

Ido Roll, Vincent Aleven, Bruce McLaren, Kenneth Koedinger


Students often use available help facilities in an unproductive fashion. To improve students’ help-seeking behavior we built the Help Tutor – a domain-independent agent that can be added as an adjunct to Cognitive Tutors. Rather than making help-seeking decisions for the students, the Help Tutor teaches better help-seeking skills by tracing students actions on a (meta)cognitive help-seeking model and giving students appropriate feedback. In a classroom evaluation the Help Tutor captured help-seeking errors that were associated with poorer learning and with poorer declarative and procedural knowledge of help seeking. Also, students performed less help-seeking errors while working with the Help Tutor. However, we did not find evidence that they learned the intended help-seeking skills, or learned the domain knowledge better. A new version of the tutor that includes a self-assessment component and explicit help-seeking instruction, complementary to the metacognitive feedback, is now being evaluated.


  • Help Avoidance:
  • Help Abuse:
  • Bottom out hint:
  • Metacognition
  • Gaming the system

Research question

Can conventional and well-established instructional principles in the domain level be used to tutor metacognitive KC such as Help Seeking KC?

Background and Significance

Not only that teaching metacognition holds the promise of improving current learning of the domain of interest, but also, or even mainly, it can accelerate future learning and successful regulation of independent learning. However, considerable evidence shows that metacognitive knowledge components are in need of better support. For example, while working with Intelligent Tutoring Systems (ITS), students try to "game the system" or do not self-explain enough. This study focuses on students' Help Seeking behavior. While students already have knowledge components which regulate their help seeking behavior, these are often faulty or shallow. For example, students often have the following shallow Help Seeking KC:

Faulty procedural KC: Cognitive aspects:

 If I don’t know the answer => 
 I should guess

Motivational aspects:

 If I get the answer correct =>
 I achieved the goal

Social aspects:

 If I ask for help =>
 I am weak

Faulty declarative KC:

 Asking for hints will always reduce my skill level
 Making an error is better than asking for a hint
 “Men never ask for help”

Rather than making the metacogntive decisions for the students (for example, by preventing help-seeking errors or gaming opportunities), this study focuses on helping students refine the Help Seeking KC and acquire better feature validly.

This study has two main contributions: - It informs us about tutoring help-seeking KC - It investigates whether conventional means and established methods can be used to effectively teach metacognitive KC.

Independent Variables

The study evaluates three independent variables: - On time feedback - direct and immediate feedback on Help Seeking errors - Explicit declarative help-seeking instruction - Preparatory self-assessment episodes, to help students identify their knowledge deficits.

Dependent variables

The study uses two levels of dependent measures: - Directly assessing Help Seeking skills - Indirectly assessing help-seeking skills through their contribution to domain learning

Direct help seeking assessment:

- Procedural knowledge - by tracings students' actions against a model of ideal help-seeking behavior

- Declarative knowledge - using hypothetical help-seeking dilemmas. - Long term retention - by assessing help-seeking behavior on subsequent learning events (was not done yet) - Transfer - assessing help-seeking behavior across environments, on a paper and pencil test which includes embedded help-seeking opportunities

Assessing help-seeking through domain knowledge - Normal posttest - Transfer measure - using not-enough-information items - Accelerate future learning - assessing learning on subsequent learning events.


The combination of explicit help-seeking instruction, on-time feedback on help seeking errors, and raising awareness to knowledge deficits will - Improve feature validity of students' help seeking skills and thus, in turn, will - Improve learning of domain knowledge by using those skills effectively.


From the first study: - Students help-seeking behavior, whether optimal or faulty, transfers well between environments. - Help Seeking feedback alone reduces help-seeking errors, but does not contribute to learning of HS skills, nor does it improve domain learning.

Analysis of data from the second study is in progress.


Possible explanations: Lack of cognitive headroom possible improvement: Preparatory self-assessment sessions Implicit feedback only Explicit Help Seeking instruction Deeply rooted behavior Longer study, across several units Faulty identification of productive learning paths Improving model based on log file analysis Ineffective hints Shorter hint sequences


Annotated bibliography

Aleven, V., & Koedinger, K.R. (2000) Limitations of student control: Do students know when they need help? in proceedings of 5th International Conference on Intelligent Tutoring Systems, 292-303. Berlin: Springer Verlag. Aleven, V., McLaren, B.M., Roll, I., & Koedinger, K.R. (2004) Toward tutoring help seeking - Applying cognitive modeling to meta-cognitive skills . in proceedings of 7th Int C on Intelligent Tutoring Systems, 227-39. Berlin: Springer-Verlag. Aleven, V., Roll, I., McLaren, B.M., Ryu, E.J., & Koedinger, K.R. (2005) An architecture to combine meta-cognitive and cognitive tutoring: Pilot testing the Help Tutor. in proceedings of 12th Int C on Artificial Intelligence in Education, Amsterdam, The Netherlands: IOS press. Aleven, V., McLaren, B.M., Roll, I., & Koedinger, K.R. (2006). Toward meta-cognitive tutoring: A model of help seeking with a Cognitive Tutor. Int J of Artificial Intelligence in Education(16), 101-30 Roll, I., Aleven, V., & Koedinger, K.R. (2004) Promoting Effective Help-Seeking Behavior through Declarative Instruction. in proceedings of 7th Int C on Intelligent Tutoring Systems, 857-9. Berlin: Springer-Verlag. Roll, I., Baker, R.S., Aleven, V., McLaren, B.M., & Koedinger, K.R. (2005) Modeling Students’ Metacognitive Errors in Two Intelligent Tutoring Systems. in L. Ardissono, (Eds.), in proceedings of User Modeling 2005, 379-88. Berlin: Springer-Verlag. Roll, I., Ryu, E., Sewall, J., Leber, B., McLaren, B.M., Aleven, V., & Koedinger, K.R. (2006) Towards Teaching Metacognition: Supporting Spontaneous Self-Assessment. in proceedings of 8th Int C on Intelligent Tutoring Systems, 738-40. Berlin: Springer Verlag. Roll, I., Aleven, V., McLaren, B.M., Ryu, E., Baker, R.S., & Koedinger, K.R. (2006) The Help Tutor: Does Metacognitive Feedback Improves Students' Help-Seeking Actions, Skills and Learning? in proceedings of 8th Int C on Intelligent Tutoring Systems, 360-9. Berlin: Springer Verlag.