Difference between revisions of "In vivo comparison of Cognitive Tutor Algebra using handwriting vs typing input"

Lisa Anthony, Jie Yang, Kenneth R. Koedinger

Summary Table

 PIs Lisa Anthony, Jie Yang, & Ken Koedinger Other Contributers n/a Study Start Date April 11, 2007 Study End Date May 25, 2007 LearnLab Site Central Westmoreland Career & Technology Center (CWCTC) and Wilkinsburg High School LearnLab Course Algebra Number of Students est. 102 Total Participant Hours est. 300 DataShop To be completed when study ends

Abstract

This in vivo classroom experiment compared differences in learning that occur depending on the modality of input during algebra equation solving. The key to this study was that the interface used was the normal Cognitive Tutor Algebra equation solver that students normally use in their classroom.

The hypothesis of this study was that, in addition to previously seen usability advantages of handwriting over typing in terms of speed and user satisfaction, handwriting would also provide learning advantages. We hypothesize two interrelated factors would be responsible for these advantages: (1) the improved support of handwriting for 2D mathematics notations such as fractions and exponents which can be difficult to represent and manipulate via the keyboard; and (2) the decrease in extraneous and irrelevant cognitive load due to removing the overhead a cumbersome menu-based interface for mathematics can provide.

Preliminary results indicate that the handwriting students finished in about half the time that the keyboard students took (14.7 minutes vs 27.0 minutes) and yet they performed just as well on the post-test. More detailed analyses are in progress on isolating the effects of modality on learning rate and/or learning efficiency.

Glossary

Forthcoming, but will probably include

• Sample worked-out-example and/or screenshot of interaction in handwriting and typing
• Learning rate/efficiency

Research question

How is robust learning affected by the modality of the generated input of students, specifically comparing handwriting and typing?

Background & Significance

Prior work has found that handwriting can be faster and more liked by users than using a keyboard and mouse for entering mathematics on the computer [1]. Anecdotal evidence suggests that students take a long time to learn an interface, possibly because it interferes with learning the goal concept. If handwriting can be shown to provide robust learning gains over traditional interfaces for mathematics, it may be possible to improve intelligent tutoring systems for mathematics by incorporating handwriting interfaces; students will be faster, more engaged and more deeply involved in knowledge construction during the learning process.

Independent Variables

Three factors were varied:

• Modality of input: free-form handwriting space vs keyboard-and-mouse solver interface
• Type of instruction: pure problem-solving vs problem-solving plus worked examples
• Type of feedback: step-targeted vs answer-targeted

Hypothesis

The handwriting modality has been shown to be faster than typing for mathematics [1], and this corresponding speed-up in the classroom implies that more detailed study of current topics or further study of more advanced topics is possible than students otherwise would be able to achieve. In addition, students' cognitive overhead during writing should be less than typing, in which they must spend time to think about how to generate the desired input, whereas in handwriting this would come more naturally due to long practice.

Dependent variables

• Near transfer, immediate: Students were given a 20-minute post-test after their sessions with the computer tutor had concluded.
• Near transfer, retention: 3 weeks after the students complete Unit 18 for the study, they will be given a 20-minute retention test consisting of problems isomorphic to those seen in the session.
• Far transfer: Far transfer items such as 4-step problems were included on all tests.
• Acceleration of future learning: We intend to analyze the log data from the students' Cognitive Tutor usage in the equation solving unit that followed the 3-step problems, to determine if there were learning curve differences during training.

Findings

Final findings in progress.

Explanation

This study is part of both the Refinement and Fluency and the Coordinative Learning clusters.

Coordinative Learning

nd addresses two of the 9 core assumptions: (1) fluency from basics: for true fluency, higher level skills must be grounded on well-practiced lower level skills; and (2) immediacy of feedback: a corollary of the emphasis on in vivo evaluation, scheduling, and explicit instruction is the idea that immediate feedback, which is a strong point of computerized instruction, facilitates learning.

The fluency from basics element in this study is relevant to the idea that students and teachers use handwritten notations in math class extensively on paper tests and when working on the chalkboard. Learning a new interface is not the goal of a math classroom, but rather learning the concepts and operations is. Thus, extraneous cognitive load of students is increased while learning the interface and learning the math conpete for resources.

The immediacy of feedback issue is not present in this study but rather in the overall project which doesn't have a node yet.

Not sure if this should be both CL and RF or just CL or just RF. We don't have two instructional activities or sources of information as CL requires...

None.

Annotated Bibliography

Analysis and write-up in progress.

References

[1] Anthony, Lisa; Yang, Jie; Koedinger, Kenneth R. (2005) "Evaluation of Multimodal Input for Entering Mathematical Equations on the Computer." ACM Conference on Human Factors in Computing Systems (CHI 2005), Portland, OR, 4 Apr 2005, pp. 1184-1187.

[2] Anthony, Lisa; Yang, Jie; Koedinger, Kenneth R. (2007) "Benefits of Handwritten Input for Students Learning Algebra Equation Solving." To appear in Proceedings of International Conference on Artificial Intelligence in Education (AIEd 2007).

Further Information

Plans for June 2007-December 2007
• Analyze data to determine effect of modality as mitigated by potential benefits of worked examples or potential drawbacks of answer-targeted feedback.
• Write up results for publication in a learning science conference.
• Based on results of this study, handwriting recognition enhancements will be performed and a summative evaluation of the prototype Handwriting Algebra Tutor will be conducted in vivo in 2007-2008.