Lab study proof-of-concept for handwriting vs typing input for learning algebra equation-solving

Lisa Anthony, Jie Yang, Kenneth R. Koedinger

Summary Table

PIs	Lisa Anthony, Jie Yang, & Ken Koedinger
Other Contributors	Research Programmers/Associates: Thomas Bolster (Research Associate, CMU HCII)
Study Start Date	August 1, 2005
Study End Date	October 8, 2005
LearnLab Site	n/a
LearnLab Course	n/a
Number of Students	48
Total Participant Hours	1200
DataShop	No

Abstract

This laboratory experiment compared differences in learning that occur depending on the modality of input during algebra equation solving. Students copied and studied a worked-out algebra example line by line before then solving an analogous problem while referring to the example. One-third of the students entered their input into a plain text box (keyboard condition), another third entered their input into a blank writing space (handwriting condition), and the final third entered their input in the writing space while also speaking the steps out loud (handwriting-plus-speaking).

The hypothesis of this study was that, in addition to previously seen usability advantages of handwriting over typing in terms of speed and user satisfaction, handwriting would also provide learning advantages. We hypothesize two interrelated factors would be responsible for these advantages: (1) the improved support of handwriting for 2D mathematics notations such as fractions and exponents which can be difficult to represent and manipulate via the keyboard; and (2) the decrease in extraneous and irrelevant cognitive load due to removing the overhead a cumbersome menu-based interface for mathematics can provide.

Preliminary results indicate that the handwriting students finished in about half the time that the keyboard students took (14.7 minutes vs 27.0 minutes) and yet they performed just as well on the post-test. More detailed analyses are in progress on isolating the effects of modality on learning rate and/or learning efficiency.

Glossary

Forthcoming, but will probably include

• Sample worked-out-example:

• Learning rate/efficiency

Research question

How is robust learning affected by the modality of the generated input of students, specifically comparing handwriting and typing?

Background & Significance

Prior work has found that handwriting can be faster and more liked by users than using a keyboard and mouse for entering mathematics on the computer [1]. Anecdotal evidence suggests that students take a long time to learn an interface, possibly because it interferes with learning the goal concept. If handwriting can be shown to provide robust learning gains over traditional interfaces for mathematics, it may be possible to improve intelligent tutoring systems for mathematics by incorporating handwriting interfaces; students will be faster, more engaged and more deeply involved in knowledge construction during the learning process.

Independent Variables

One independent variable was used:

• Modality of input: handwriting, typing, or handwriting-plus-speaking.

Hypothesis

The handwriting modality has been shown to be faster than typing for mathematics [1], and this corresponding speed-up in the classroom implies that more detailed study of current topics or further study of more advanced topics is possible than students otherwise would be able to achieve. In addition, students' cognitive overhead during writing should be less than typing, in which they must spend time to think about how to generate the desired input, whereas in handwriting this would come more naturally due to long practice.

Dependent variables

• Near transfer, immediate: During training, examples alternated with problems, and the problems were solved in one of the 3 modalities/conditions. Each problem was similar to the example that preceded it, so performance on it is a measure of normal learning (near transfer, immediate testing). Analyses of log data to determine error rate during training are in progress of being analyzed.

• Near transfer, retention: After the session the students were given a 20-minute post-test consisting of problems isomorphic to those seen in the session. Handwriting students and typing students both achieved similar pre-post gains, but handwriting-plus-speaking students achieved much lower gains.

• Far transfer: No far transfer items were included.

• Acceleration of future learning: No acceleration of future learning measures were included in this laboratory study.

Findings

Results from this study showed that students in the handwriting condition finished the curriculum in half the time of their typing counterparts (F(2,35)=11.05, p<0.0005). Yet there was no significant difference in their pre-to-post scores between conditions (F(2,35)=0.293, n.s.). Students appear to have learned just as much in about half the time! In a classroom situation, this would allow teachers to give students more practice or move on to more advanced material in the curriculum sooner.

There was also a significant interaction between modality and the appearance of fractions in a problem (F2,36=5.25, p<0.01), which implies that the advantages we’ve seen for handwriting only improve as the math gets more complex.

In their own words, students commented that handwriting “made it easier” and “takes a shorter time”—statements that lend support to the hypothesis that handwriting involves less extraneous cognitive load. While this is only a preliminary result, we plan to explore this further in later studies by including a structured self-report of student-perceived cognitive load, modeled after (Paas & Van Merrienboer, 1994), in which they asked students to rate their perceived amount of mental effort during various instructional paradigms.

In addition to these learning-related results, we found that speed differed by modality: students were two times faster in handwriting than in typing to complete the problem set given to them. Students also rated the handwriting condition more highly than the typing condition (70% chose it as their favorite modality), after having copied a set of given equations in both conditions.

Explanation

This study is part of the Refinement and Fluency cluster (was Coordinative Learning) and addresses one of the 9 core assumptions: (1) fluency from basics: for true fluency, higher level skills must be grounded on well-practiced lower level skills.

The fluency from basics element in this study is relevant to the idea that students and teachers use handwritten notations in math class extensively on paper tests and when working on the chalkboard. Learning a new interface is not the goal of a math classroom, but rather learning the concepts and operations is. Thus, extraneous cognitive load of students is increased while learning the interface and learning the math compete for resources.

Descendants

None.

Annotated Bibliography

[1] Anthony, Lisa; Yang, Jie; Koedinger, Kenneth R. (2005) "Evaluation of Multimodal Input for Entering Mathematical Equations on the Computer." ACM Conference on Human Factors in Computing Systems (CHI 2005), Portland, OR, 4 Apr 2005, pp. 1184-1187.

[2] Anthony, Lisa; Yang, Jie; Koedinger, Kenneth R. (2007) "Benefits of Handwritten Input for Students Learning Algebra Equation Solving." To appear in Proceedings of International Conference on Artificial Intelligence in Education (AIEd 2007).

Further Information

Plans for June 2007-December 2007

• This study has been completed, analyzed, submitted, and accepted for publication.
• The next steps in this line of research are relevant to In vivo comparison of Cognitive Tutor Algebra using handwriting vs typing input which is currently ongoing.