Roll help seeking principle

Brief statement of principle

Support metacognition in the context of problem solving using established principles of cognitive tutoring.

Description of principle

Instruction should yield a persistent effect on students meta-cognitive skills in a fashion that persists beyond the scope of the tutored environment. That is, the tutored practice should help students become better at learning.

This is the common effect of tutoring at the cognitive level - for example, students remember Pythagorean theorem also outside the specific instructional unit. Therefore, perhaps meta-cognitive tutoring could use established principles of cognitive tutoring.

Operational definition

Meta-cognitive tutoring should:

• Giving direct instruction
• Giving immediate feedback on errors
• Prompting for self-assessment

Experimental support

Laboratory experiment support

None that we know of.

In vivo experiment support

There is much support for supporting metacognition and reaching an effect within the tutored environment. Most notably, the work on self-explanation supports that (e.g., Aleven and Koedinger 2002; Conati and VanLehn 1999). Also, it is known that systems with hints are better than no hints, and that contingent hints are better than conventional hints. (Wood & Wood, 1999) However, these systems support meta-cognition within the scope of the system and do not thrive to affect behavior in a persistent matter, outside the tutoring environment.

So far we were not able to confirm (or dispute) this principle. Our current results show that meta-cognitive tutoring yields better meta-cognitive behavior within the tutored environment, but fail to generate a persistent effect (i.e., better meta-cognitive behavior in a novel context) or better domain learning (as evaluated using pre- and post-tests). Please see the project page for more details.

Theoretical rationale

(These entries should link to one or more learning processes.)

In short, Metacognitive tutoring is important since better metacognition knowledge should help students make sense of the material to be learned. Since metacognitive knowledge is type of knowledge, it should be doable to tutor it in a similar fashion to tutoring domain knowledge.

Conditions of application

So far we were not able to achieve the desired effect under any conditions.

It is hypothesized that such tutoring is doable with upper-achieving students.

Caveats, limitations, open issues, or dissenting views

Our inability to achieve the desired effect has probably to do with the differences between domain and metacognitive tutoring. Here is an exert from a recent publication: "Students’ tendency to skip hints suggests that perhaps the main issue is not lack of knowledge, but lack of motivation. For students who ignore intermediate hints, metacognitive messages offer little incentive. While the HSSE can increase the probability that a proper hint level appears on the screen, it has no influence on whether it is being read. Students may ignore the messages for several reasons. For example, they may habitually click through hints, and may resent the changes that the HSSE imposes. This idea is consistent with the teachers’ observation that the students were not fond of the HSSE error messages. The test data discussed above provides support for this idea. On 7 out of the 12 hint evaluations students scored lower on items with hints than on items with no hints. A cognitive load explanation does not account for this difference, since the Request hints did not add much load. A more likely explanation is that students chose to skip the hints since they were new to them in the given context. Baker [5] reviewed several reasons for why students game the system. While no clear answer was given, the question is applicable here as well. Pintrich [9] suggests that while appropriate motivation facilitates the use of existing metacognitive skills, other motivations may hinder such productive behavior. Motivational issues bring us to our final hypothesis. Time Preference Discount is a term coined in economics that describes behavior in which people would rather have a smaller immediate reward over a distant greater reward. In the tutoring environment, comparing the benefit of immediate correct answer with the delayed benefit (if any) of acting in a metacognitively correct manner may often lead the student to choose the first. If that is indeed the case, then students may already have the right metacognitive skills in place. The question we should be asking ourselves is not only how to get students to learn the desired metacognitive skills – but mainly, how to get students to use them. "

(Roll, I., Aleven, V., McLaren, B. M., & Koedinger, K. R. (2007). Can help seeking be tutored? Searching for the secret sauce of metacognitive tutoring. International Conference on Artificial Intelligence in Education, , 203-10.) [pdf]