REAP Study on Word Sense Disambiguation (Summer 2007)

REAP Study on Word Sense Disambiguation (Summer 2007)

Logistical Information

Contributors	Maxine Eskenazi, Alan Juffs, Anagha Kulkarni, Jamie Callan, Michael Heilman
Study Start Date	May, 2007
Study End Date	July, 2007
Learnlab Courses	English Language Institute Reading 4&5 (ESL LearnLab)
Number of Students	~45
Total Participant Hours (est.)	~250
Data in Datashop	no

Abstract

In previous REAP studies, there has been no control over the sense of the word being taught to students. This lack of control could be a problem for the many words that are polysemous (e.g., "bank", "initial", "labor", "prime"). Word sense disambiguation (WSD) techniques may be a means of providing control over word senses in REAP readings, practice exercises, and assessments. However, current WSD technologies have a significant error rate, correctly identifying around two thirds of senses correctly on average.

In this study, students will be randomly assigned to control or treatment conditions. Students in the treatment condition will receive readings and practice materials that are matched, using WSD, with the target senses of vocabulary words. These senses will be chosen by course teachers (for example, a course teacher would likely want the sense of "prime" that means, roughly, "of great importance or first in rank" rather than the sense related to mathematics (e.g., "prime numbers"). The set of words on which REAP will provide instruction will be constructed from words that have multiple senses. Data from previous studies will also be examined to constrain the set of words to be those which students have found difficult (i.e., have performed poorly even after receiving instruction).

Research question

Does automatically matching the sense of vocabulary words to the target sense chosen by teachers improve learning?

Hypotheses

For certain polysemous words with distinct senses (e.g., "prime"), matching of training materials to target senses will improve robust learning measures.