|
|
| (One intermediate revision by the same user not shown) |
| Line 3: |
Line 3: |
| | === Abstract === | | === Abstract === |
| | | | |
| − | Much work has been done in the field on modality of presentation: that is, how to best present instructional materials to students, both in the classroom and on the computer. This project’s focus is instead of modality of generation: how the modality in which students produce their solutions interacts with the cognitive processes involved in conceptualizing and working through a solution. Most intelligent tutoring systems have relied on standard computer interface paradigms using the keyboard and mouse, even in mathematics. These interfaces are often idiosyncratic and quirky in their support for standard math notations such as two-dimensional constructs like fractions and exponents. The spatial relationships between characters has inherent and important meaning in mathematics, far more so than in other forms of writing, yet computer typing interfaces are ill-equipped to deal with them. Freeform handwriting input may be uniquely suited to entering mathematics input on the computer by virtue of its natural support for spatially variegated notations and its lack of constraints imposed on students producing a solution (allowing the problem-solving process to benefit from foundational fluency in handwriting). | + | Much work has been done in the field on modality of <b>presentation</b>: that is, how to best present instructional materials to students, both in the classroom and on the computer. This project’s focus is instead of modality of <b>generation</b>: how the modality in which students produce their solutions interacts with the cognitive processes involved in conceptualizing and working through a solution. |
| − | Therefore, this project’s goal is to explore the ways in which the use of handwriting recognition in the interfaces of intelligent tutoring systems, already shown to be highly effective learning environments, can lead to improved learning gains. Our work has already shown that handwriting provides usability benefits in that speed of entry increases, user error decreases, and user satisfaction increases. We have also shown preliminary evidence that handwriting also provides learning benefits, in that students solving the same problems by handwriting as others who were typing took only half the time as their typing counterparts.
| |
| | | | |
| − | One concern with the use of handwriting in intelligent tutoring systems, however, is that recognition technology is not perfect. To the extent that we cannot be confident of correctly recognizing what the student is writing, we cannot provide detailed, step-targeted feedback. Therefore, a trade-off is clear between development effort to improve recognition accuracy and need to support step-targeted feedback. We attempt to address this trade-off by using worked examples, providing a sort of feed-forward instead and providing opportunities for students to use coordination of the given problem and the given example to improve their knowledge. To this end, this project consists of a number of studies that explore what the advantages of using handwriting are, what cognitive and learning factors contribute to these advantages, and how we can leverage these advantages in real tutoring systems with minimal technical development cost.
| + | Most intelligent tutoring systems have relied on standard computer interface paradigms using the keyboard and mouse, even in mathematics. These interfaces are often idiosyncratic and quirky in their support for standard math notations such as two-dimensional constructs like fractions and exponents. The spatial relationships between characters has inherent and important meaning in mathematics, far more so than in other forms of writing, yet computer typing interfaces are ill-equipped to deal with them. Freeform handwriting input may be uniquely suited to entering mathematics input on the computer by virtue of its natural support for spatially variegated notations and its lack of constraints imposed on students producing a solution (allowing the problem-solving process to benefit from foundational fluency in handwriting). |
| − |
| |
| − | === Glossary ===
| |
| | | | |
| − | === Research Question ===
| + | Therefore, this project’s goal is to explore the ways in which the use of handwriting recognition in the interfaces of intelligent tutoring systems, already shown to be highly effective learning environments, can lead to improved learning gains. Our work has already shown that handwriting provides usability benefits in that speed of entry increases, user error decreases, and user satisfaction increases. We have also shown preliminary evidence that handwriting also provides learning benefits, in that students solving the same problems by handwriting as others who were typing took only half the time as their typing counterparts. |
| − | | |
| − | === Background/Significance ===
| |
| − | | |
| − | === Dependent Variables ===
| |
| − | | |
| − | === Independent Variables ===
| |
| − | | |
| − | === Hypotheses ===
| |
| − | | |
| − | === Findings ===
| |
| − | This project has overseen three studies since it began in November 2004. The first study, the Math Input Study, was a motivating study designed to explore what advantages, if any, handwriting-based input has for mathematics entry on the computer. Learning in a classroom setting is characterized by constraints on time, varying student motivation and engagement, and other factors not directly related to test scores. Our study showed that students who entered math equations via handwriting input were three times faster, were less prone to errors in input, and enjoyed their experience more. In the classroom, this can translate to increased depth or breadth of coverage by virtue of the extra time afforded, and to improve student motivation by virtue of their increased engagement. See (Anthony et al, 2005) for more details.
| |
| − | | |
| − | The second study, the Preliminary Learning Study, took the first study’s results one step further and applied handwriting-based input to a learning situation. In this study we compared students solving problems in a simple type-in interface with a handwriting input space; instruction was in the form of worked examples interspersed with problem solving, and feedback was answer-only (“Correct”/“Incorrect”). This was a laboratory study to determine whether or not novice math students engaged in a learning task would experience the same positive effects of using a handwriting interface over a keyboard one. In fact, students were faster: two times faster in handwriting than in typing to complete the problem set given to them. Students also rated the handwriting condition more highly than the typing condition (70% chose it as their favorite modality), after having copied a set of given equations in both conditions.
| |
| − | | |
| − | Results from this study showed that students in the handwriting condition finished the curriculum in half the time of their typing counterparts (Figure 1a, F2,35=11.05, p<0.0005). Yet there was no significant difference in their pre-to-post scores between conditions (Figure 1b, F2,35=0.293, n.s.). Students appear to have learned just as much in about half the time! In a classroom situation, this would allow teachers to give students more practice or move on to more advanced material in the curriculum sooner. There was also a significant interaction between modality and the appearance of fractions in a problem (F2,36=5.25, p<0.01), which implies that the advantages we’ve seen for handwriting only improve as the math gets more complex. In their own words, students commented that handwriting “made it easier” and “takes a shorter time”—statements that lend support to the hypothesis that handwriting involves less extraneous cognitive load. While this is only a preliminary result, we plan to explore this further in later studies by including a structured self-report of student-perceived cognitive load, modeled after (Paas & Van Merrienboer, 1994), in which they asked students to rate their perceived amount of mental effort during various instructional paradigms.
| |
| − | | |
| − | The third study, the Worked Examples Study, is currently ongoing. It is an in vivo study taking place at a local LearnLab school, CWCTC. Intelligent tutoring systems such as Cognitive Tutors have long incorporated directed step-by-step feedback throughout the problem-solving process; while this is considered to be a strength of the method, it has not been shown to be critical to effective learning. If such detailed feedback is not necessary for student success, the instructional paradigm can be altered when using handwriting input to prevent recognition errors from
| |
| − |
| |
| − | Figure 1a. Time-on-task differences between the handwriting and typing conditions in the Preliminary Learning Study. Handwriting was two times faster than typing. Figure 1b. Learning as measured by gain from pre-test to post-test in the Preliminary Learning Study. No significant difference was seen between modalities.
| |
| − | | |
| − | | |
| − | interrupting the student. For example, we could use worked examples as a method of feed-forward to help students. The study underway is designed to begin to address this concern by comparing existing Cognitive Tutors that provide detailed feedback to the same systems that also provide worked examples during problem solving. We intend to analyze student learning as well as student hint and help-seeking during use of the tutoring system to determine whether students that are provided with worked examples use the hint facilities of the tutor less frequently.
| |
| | | | |
| − | These studies continue to inform our development of a theory of robust learning in that we have seen the first positive evidence in favor of handwriting interfaces for learning applications. Students using handwriting in the Preliminary Learning Study may have experienced a heightened sense of fluency with the mathematics they were learning, in that the interface was better able to allow them to more directly represent and manipulate the equations. In their own words, students praised handwriting for similar reasons. However, this needs to be isolated in greater detail in future studies. Eventually, we expect that students using our system will be able to both achieve greater fluency and engage in robust coordinative learning, that is, integrating the feedforward worked examples instruction along with the step-by-step directed feedback in order to construct a deeper understanding of the target concepts.
| + | One concern with the use of handwriting in intelligent tutoring systems, however, is that recognition technology is not perfect. To the extent that we cannot be confident of correctly recognizing what the student is writing, we cannot provide detailed, step-targeted <b>feedback</b>. Therefore, a trade-off is clear between development effort to improve recognition accuracy and need to support step-targeted feedback. We attempt to address this trade-off by using <b>worked examples</b>, providing a sort of feed-forward instead and providing opportunities for students to use coordination of the given problem and the given example to improve their knowledge. To this end, this project consists of a number of studies that explore what the advantages of using handwriting are, what cognitive and learning factors contribute to these advantages, and how we can leverage these advantages in real tutoring systems with minimal technical development cost. |
| | | | |
| | === Descendents === | | === Descendents === |
Lisa Anthony, Jie Yang, Kenneth R. Koedinger
Abstract
Much work has been done in the field on modality of presentation: that is, how to best present instructional materials to students, both in the classroom and on the computer. This project’s focus is instead of modality of generation: how the modality in which students produce their solutions interacts with the cognitive processes involved in conceptualizing and working through a solution.
Most intelligent tutoring systems have relied on standard computer interface paradigms using the keyboard and mouse, even in mathematics. These interfaces are often idiosyncratic and quirky in their support for standard math notations such as two-dimensional constructs like fractions and exponents. The spatial relationships between characters has inherent and important meaning in mathematics, far more so than in other forms of writing, yet computer typing interfaces are ill-equipped to deal with them. Freeform handwriting input may be uniquely suited to entering mathematics input on the computer by virtue of its natural support for spatially variegated notations and its lack of constraints imposed on students producing a solution (allowing the problem-solving process to benefit from foundational fluency in handwriting).
Therefore, this project’s goal is to explore the ways in which the use of handwriting recognition in the interfaces of intelligent tutoring systems, already shown to be highly effective learning environments, can lead to improved learning gains. Our work has already shown that handwriting provides usability benefits in that speed of entry increases, user error decreases, and user satisfaction increases. We have also shown preliminary evidence that handwriting also provides learning benefits, in that students solving the same problems by handwriting as others who were typing took only half the time as their typing counterparts.
One concern with the use of handwriting in intelligent tutoring systems, however, is that recognition technology is not perfect. To the extent that we cannot be confident of correctly recognizing what the student is writing, we cannot provide detailed, step-targeted feedback. Therefore, a trade-off is clear between development effort to improve recognition accuracy and need to support step-targeted feedback. We attempt to address this trade-off by using worked examples, providing a sort of feed-forward instead and providing opportunities for students to use coordination of the given problem and the given example to improve their knowledge. To this end, this project consists of a number of studies that explore what the advantages of using handwriting are, what cognitive and learning factors contribute to these advantages, and how we can leverage these advantages in real tutoring systems with minimal technical development cost.
Descendents
Completed Experiments
In-Progress Experiments
Planned Experiments
- Summative evaluation of enhanced handwriting interface with step-targeted feedback
Annotated Bibliography
[1] Anthony, Lisa; Yang, Jie; Koedinger, Kenneth R. (2005) "Evaluation of Multimodal Input for Entering Mathematical Equations on the Computer." ACM Conference on Human Factors in Computing Systems (CHI 2005), Portland, OR, 4 Apr 2005, pp. 1184-1187.
[2] Anthony, Lisa; Yang, Jie; Koedinger, Kenneth R. (2007) "Benefits of Handwritten Input for Students Learning Algebra Equation Solving." To appear in Proceedings of International Conference on Artificial Intelligence in Education (AIEd 2007).
References
Further Information
