Theory Wiki - User contributions [en]

REAP Study on the Correlation Between Automatically and Manually Generated Reading Comprehension Questions (Summer 2007)

2007-07-25T22:36:17Z

Feeney:

== REAP Study on the Correlation Between Automatically and Manually Generated Reading Comprehension Questions (Summer 2007) ==

=== Logistical Information ===

{| border="1"
|+
|-
| '''Contributors''' || Christine M. Feeney and Michael Heilman
|-
| '''Study Start Date''' || June, 2007
|-
| '''Study End Date''' || July, 2007
|-
| '''Learnlab Courses''' || no
|-
| '''Number of Students''' || 30
|-
| '''Total Participant Hours (est.)''' || 300
|-
| '''Data in Datashop''' || no
|}

=== Abstract ===

Previous psychological research has identified two types of comprehension: shallow, in which people can reproduce information, and deep, in which people can comprehend the meaning of information. In addition, researchers have found that these two types of comprehension are separate and disparate. The PSLC currently employs an English as a Second Language vocabulary tutor, REAP, that uses shallow, automatically generated, comprehension questions to check that students are actively reading, rather than just skimming, practice reading passages. The purpose of the current study was to examine performance on the REAP-type questions with manually authored, deeper, reading comprehension questions. Participants were thirty undergraduate students (male = 11) participating in summer research programs at Carnegie Mellon University. The researcher predicted a positive correlation between the two testing types, which was supported by the data (r = .366, p < 0.0005, one-tailed t-test).

=== Glossary ===

''Shallow Comprehension:'' Processing directed towards reproducing the learning material without necessarily understanding it.

''Deep Comprehension:'' Processing directed towards comprehending the intended meaning of the learning material.

=== Research question ===

Does performance on the shallow, automatically-generated questions correlate with performance on more sophisticated, deep reading comprehension questions?

=== Dependent variables ===

Percentage of Shallow Comprehension Questions Answered Correctly

Percentage of Deep Comprehension Questions Answered Correctly

=== Independent variables ===

Five reading passages, each of which was followed by four shallow reading comprehension questions and three to five deep reading comprehension questions.

=== Hypothesis ===

A significant positive correlation between shallow and deep reading comprehension questions will occur in this study.

=== Explanation ===

There was a relationship between performance on the two question types (automatically-generated shallow reading comprehension questions and manually-generated deep comprehension questions), which could imply that using the automatically-generated questions as a check for reading comprehension for the REAP program is a sufficient measure (r = .366, p < 0.0005, one-tailed test). Further studies could examine different types of deep reading comprehension questions, removing the “giveaway words” from the automatically-generated distracters, and examining whether ESL students approach these questions in a different manner than native English speakers.

=== Descendents ===

=== Notes ===

The researchers chose five passages of between five hundred to one thousand words because this length is comparable to reading comprehension passages that appear on standardized tests, such as the GRE, and to REAP texts. The researchers used a variety of sources in order to have passages of varying difficulty.

=== Annotated bibliography ===

Curran, James R. Curran & Moens, Marc. (2002a). Improvements in automatic thesaurus extraction. Proceedings of the Workshop of the ACL Special Interest Group on the Lexicon (SIGLEX), 59-66.

Davoudi, Mohammad. (2005). Inference Generation Skill and Text Comprehension. The Reading Matrix, 5(1), 106-123.

Heilman, M., Collins-Thompson, K., Callan, J. & Eskenazi, M. (2006). Classroom success of an Intelligent Tutoring System for lexical practice and reading comprehension. Proceedings of the Ninth International Conference on Spoken Language Processing.

Heilman, M. & Eskenazi, M. In Press. Application of Automatic Thesaurus Extraction for Computer Generation of Vocabulary Questions. Proceedings of the SLaTE Workshop on Speech and Language Technology in Education.

REAP Study on the Correlation Between Automatically and Manually Generated Reading Comprehension Questions (Summer 2007)

2007-07-25T22:35:41Z

Feeney:

REAP Study on the Correlation Between Automatically and Manually Generated Reading Comprehension Questions (Summer 2007)

2007-07-25T22:21:44Z

Feeney:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en" dir="ltr">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="keywords" content=" REAP Study on the Correlation Between Automatically and Manually Generated Reading Comprehension Questions (Summer 2007) " />
<link rel="shortcut icon" href="/favicon.ico" />
<title> REAP Study on the Correlation Between Automatically and Manually Generated Reading Comprehension Questions (Summer 2007) - Pslc</title>
<style type="text/css" media="screen,projection">/*<![CDATA[*/ @import "/research/wiki/skins/monobook/main.css?7"; /*]]>*/</style>
<link rel="stylesheet" type="text/css" media="print" href="/research/wiki/skins/common/commonPrint.css" />





<script type="text/javascript">var skin = 'monobook';var stylepath = '/research/wiki/skins';</script>
<script type="text/javascript" src="/research/wiki/skins/common/wikibits.js"></script>
<script type="text/javascript" src="/research/wiki/index.php?title=-&action=raw&smaxage=0&gen=js"></script>
<style type="text/css">/*<![CDATA[*/
@import "/research/wiki/index.php?title=MediaWiki:Common.css&action=raw&ctype=text/css&smaxage=18000";
@import "/research/wiki/index.php?title=MediaWiki:Monobook.css&action=raw&ctype=text/css&smaxage=18000";
@import "/research/wiki/index.php?title=-&action=raw&gen=css&maxage=18000";
/*]]>*/</style>

</head>
<body class="ns-0 ltr">
<div id="globalWrapper">
<div id="column-content">
<div id="content">
<a name="top" id="top"></a>
<h1 class="firstHeading"> REAP Study on the Correlation Between Automatically and Manually Generated Reading Comprehension Questions (Summer 2007) </h1>
<div id="bodyContent">
<h3 id="siteSub">From Pslc</h3>
<div id="contentSub"></div>
<div id="jump-to-nav">Jump to: <a href="#column-one">navigation</a>, <a href="#searchInput">search</a></div> 
<table id="toc" class="toc" summary="Contents"><tr><td><div id="toctitle"><h2>Contents</h2></div>
<ul>
<li class='toclevel-1'><a href="# Study_on_the_Correlation_Between_Automatically_and_Manually_Generated_Reading_Comprehension_Questions"><span class="tocnumber">1</span> <span class="toctext"> REAP Study on the Correlation Between Automatically and Manually Generated Reading Comprehension Questions </span></a>
<ul>
<li class='toclevel-2'><a href="#Logistical_Information"><span class="tocnumber">1.1</span> <span class="toctext">Logistical Information</span></a></li>
<li class='toclevel-2'><a href="#Abstract"><span class="tocnumber">1.2</span> <span class="toctext">Abstract</span></a></li>
<li class='toclevel-2'><a href="#Glossary"><span class="tocnumber">1.3</span> <span class="toctext">Glossary</span></a></li>
<li class='toclevel-2'><a href="#Research_question"><span class="tocnumber">1.4</span> <span class="toctext">Research question</span></a></li>
<li class='toclevel-2'><a href="#Dependent_variables"><span class="tocnumber">1.5</span> <span class="toctext">Dependent variables</span></a></li>
<li class='toclevel-2'><a href="#Independent_variables"><span class="tocnumber">1.6</span> <span class="toctext">Independent variables</span></a></li>
<li class='toclevel-2'><a href="#Hypotheses"><span class="tocnumber">1.7</span> <span class="toctext">Hypotheses</span></a></li>
<li class='toclevel-2'><a href="#Explanation"><span class="tocnumber">1.8</span> <span class="toctext">Explanation</span></a></li>
<li class='toclevel-2'><a href="#Descendents"><span class="tocnumber">1.9</span> <span class="toctext">Descendents</span></a></li>
<li class='toclevel-2'><a href="#Notes"><span class="tocnumber">1.10</span> <span class="toctext">Notes</span></a></li>
<li class='toclevel-2'><a href="#Annotated_bibliography"><span class="tocnumber">1.11</span> <span class="toctext">Annotated bibliography</span></a></li>
</ul>
</li>
</ul>
</td></tr></table><script type="text/javascript"> if (window.showTocToggle) { var tocShowText = "show"; var tocHideText = "hide"; showTocToggle(); } </script>
<div class="editsection" style="float:right;margin-left:5px;">[<a href="/research/wiki/index.php?title= REAP_Study_on_the_Correlation_Between_Automatically_and_Manually_Generated_Reading_Comprehension_Questions%28Summer_2007%29&action=edit&section=1" title="Edit section: REAP Study on the Correlation Between Automatically and Manually Generated Reading Comprehension Questions ">edit</a>]</div><a name=" REAP_Study_on_the_Correlation_Between_Automatically_and_Manually_Generated_Reading_Comprehension_Questions "></a><h2> REAP Study on the Correlation Between Automatically and Manually Generated Reading Comprehension Questions </h2>
<div class="editsection" style="float:right;margin-left:5px;">[<a href="/research/wiki/index.php?title= REAP_Study_on_the_Correlation_Between_Automatically_and_Manually_Generated_Reading_Comprehension_Questions_%28Summer_2007%29&action=edit&section=2" title="Edit section: Logistical Information">edit</a>]</div><a name="Logistical_Information"></a><h3> Logistical Information </h3>
<table border="1">
<caption>
</caption>
<tr>
<td> <b>Contributors</b> </td><td> Christine M. Feeney and Michael Heilman
</td></tr>
<tr>
<td> <b>Study Start Date</b> </td><td> June, 2007
</td></tr>
<tr>
<td> <b>Study End Date</b> </td><td> July, 2007
</td></tr>
<tr>
<td> <b>Learnlab Courses</b> </td><td>none
</td></tr>
<tr>
<td> <b>Number of Students</b> </td><td> 30
</td></tr>
<tr>
<td> <b>Total Participant Hours (est.)</b> </td><td> 300
</td></tr>
<tr>
<td> <b>Data in Datashop</b> </td><td> no
</td></tr></table>
<div class="editsection" style="float:right;margin-left:5px;">[<a href="/research/wiki/index.php?title= REAP_Study_on_the_Correlation_Between_Automatically_and_Manually_Generated_Reading_Comprehension_Questions_%28Summer_2007%29&action=edit&section=3" title="Edit section: Abstract">edit</a>]</div><a name="Abstract"></a><h3> Abstract </h3>
<p> Previous psychological research has identified two types of comprehension: shallow, in which people can reproduce information, and deep, in which people can comprehend the meaning of information. In addition, researchers have found that these two types of comprehension are separate and disparate. The PSLC currently employs an English as a Second Language vocabulary tutor, REAP, that uses shallow, automatically generated, comprehension questions to check that students are actively reading, rather than just skimming, practice reading passages. The purpose of the current study was to examine performance on the REAP-type questions with manually authored, deeper, reading comprehension questions. Participants were thirty undergraduate students (male = 11) participating in summer research programs at Carnegie Mellon University. The researcher predicted a positive correlation between the two testing types, which was supported by the data (r = .366, p < 0.0005, one-tailed t-test).
</p>
<div class="editsection" style="float:right;margin-left:5px;">[<a href="/research/wiki/index.php?title= REAP_Study_on_the_Correlation_Between_Automatically_and_Manually_Generated_Reading_Comprehension_Questions_%28Summer_2007%29&action=edit&section=4" title="Edit section: Glossary">edit</a>]</div><a name="Glossary"></a><h3> Glossary </h3>
<p><i>Shallow Comprehension:</i> Processing is directed towards reproducing the learning material without necessarily understanding it.
</p>
<p>
<i> Deep Comprehension:</i> Processing is directed towards comprehending the intended meaning of the learning material.
</p>
<div class="editsection" style="float:right;margin-left:5px;">[<a href="/research/wiki/index.php?title= REAP_Study_on_the_Correlation_Between_Automatically_and_Manually_Generated_Reading_Comprehension_Questions_%%28Summer_2007%29&action=edit&section=5" title="Edit section: Research question">edit</a>]</div><a name="Research_question"></a><h3> Research question </h3>
<p> Does performance on the shallow, automatically-generated questions correlate with performance on more sophisticated, deep reading comprehension questions?
</p>
<div class="editsection" style="float:right;margin-left:5px;">[<a href="/research/wiki/index.php?title= REAP_Study_on_the_Correlation_Between_Automatically_and_Manually_Generated_Reading_Comprehension_Questions_%28Summer_2007%29&action=edit&section=6" title="Edit section: Dependent variables">edit</a>]</div><a name="Dependent_variables"></a><h3> Dependent variables </h3>
<p> Percentage of Shallow Comprehension Questions Answered Correctly.
</p><p> Percentage of Deep Comprehension Questions Answered Correctly.
</p>
<div class="editsection" style="float:right;margin-left:5px;">[<a href="/research/wiki/index.php?title= REAP_Study_on_the_Correlation_Between_Automatically_and_Manually_Generated_Reading_Comprehension_Questions_%28Summer_2007%29&action=edit&section=7" title="Edit section: Independent variables">edit</a>]</div><a name="Independent_variables"></a><h3> Independent variable </h3>
<p>Five reading passages, each of which was followed by four shallow reading comprehension questions and three to five deep reading comprehension questions.
</p>
<div class="editsection" style="float:right;margin-left:5px;">[<a href="/research/wiki/index.php?title= REAP_Study_on_the_Correlation_Between_Automatically_and_Manually_Generated_Reading_Comprehension_Questions_%28Summer_2007%29&action=edit&section=8" title="Edit section: Hypotheses">edit</a>]</div><a name="Hypotheses"></a><h3> Hypotheses </h3>
<p> A significant positive correlation between shallow and deep reading comprehension questions will occur in this study.
</p>
<div class="editsection" style="float:right;margin-left:5px;">[<a href="/research/wiki/index.php?title= REAP_Study_on_the_Correlation_Between_Automatically_and_Manually_Generated_Reading_Comprehension_Questions_%28Summer_2007%29&action=edit&section=9" title="Edit section: Explanation">edit</a>]</div><a name="Explanation"></a><h3> Explanation </h3>
<p>The study found no significant differences between the two conditions (2 or 4 words/text). This null result may be due to statistical power and mistakes in the study design. The tutor had difficulty providing high quality readings with four vocabulary words. Thus, the tutor often had to give texts with 3 or even 2 words to students in the 4-word condition. Additionally, the optimal number of words per text (the density of practice) may be higher than 4 words. There may be a threshold at which the density of practice becomes too high. This threshold may actually be at 5 or even 10 words given the average length of the texts used by the REAP tutor. Some research suggests that up to 5% of the words in a text can be unknown before comprehension is impeded (Laufer, 1992). Although the task for that study was different (reading comprehension rather than vocabulary acquisition), it suggests that the optimal level of stretch may be such that as many as 20 words may be unknown in a text with 400 word types (which is approximately the mean number of word types in REAP texts). Other research, however, suggests higher thresholds up to 98% (Nation, 2001).
</p><p>Although the results for stretch from this study were inconclusive, a great deal of interesting data were gathered. Some interesting results have been found, and further analysis is possible.
</p><p>Many potential variables exist that could lead to successful learning of new vocabulary: general ESL ability as measured by the MTELP, time on task, number of words looked up, pre-test scores on the academic word list. A step-wise linear regression showed that the most significant predictor of acquisition of new vocabulary was the size of the student's vocabulary scores on the academic word list prior to instruction (r= 0.77, p < .0001). However, the effect of this prior knowledge was diminished by the number of texts the students read r= -0.17). Together, in the stepwise regression, these two variable accounted for 67% of the variance (r2= .67).
</p><p>This finding corroborates findings of Stanovich (1986) and James (1996) who refer to the phenomenon as the "Matthew effect" (after a Bible passage), whereby the rich get richer and the poor get poorer. In this case, it is not 'general richness' expressed in terms of higher general ESL proficiency, but specifically knowledge of words on the academic word list. In other words, the more knowledge components in the domain of acquisition a learner brings to the task, the greater the potential for future learning.
</p><p>It was also found that students frequently access dictionary definitions for non-target words that are well below their expected reading level (e.g., "hitting"). This may lead to revisions of curricula for ESL vocabulary learning.
</p>
<div class="editsection" style="float:right;margin-left:5px;">[<a href="/research/wiki/index.php?title=REAP_Study_on_Vocabulary_Stretch_%28Spring_2006%29&action=edit&section=10" title="Edit section: Descendents">edit</a>]</div><a name="Descendents"></a><h3> Descendents </h3>
<div class="editsection" style="float:right;margin-left:5px;">[<a href="/research/wiki/index.php?title=REAP_Study_on_Vocabulary_Stretch_%28Spring_2006%29&action=edit&section=11" title="Edit section: Notes">edit</a>]</div><a name="Notes"></a><h3> Notes </h3>
<p>Target vocabulary lists were determined by a lengthy pre-test using multiple choice cloze questions. Each question takes 30-60 seconds on average, depending on the student. In later studies, self-assessment pre-tests were preferred due to time constraints and initial impressions of the students. Lengthy pre-tests for determining target word lists leave a poor initial impression of the tutor.
</p>
<div class="editsection" style="float:right;margin-left:5px;">[<a href="/research/wiki/index.php?title=REAP_Study_on_Vocabulary_Stretch_%28Spring_2006%29&action=edit&section=12" title="Edit section: Annotated bibliography">edit</a>]</div><a name="Annotated_bibliography"></a><h3> Annotated bibliography </h3>
<p>Laufer, B. (1992). How much lexis is necessary for reading comprehension? In Vocabulary and Applied Linguistics, Pierre J. L. Arnaud and Henri Béjoint (eds.), 126–132. London: Macmillan.
</p>

<div class="printfooter">
Retrieved from "<a href="http://www.learnlab.org/research/wiki/index.php/REAP_Study_on_Vocabulary_Stretch_%28Spring_2006%29">http://www.learnlab.org/research/wiki/index.php/REAP_Study_on_Vocabulary_Stretch_%28Spring_2006%29</a>"</div>

<div class="visualClear"></div>
</div>
</div>
</div>
<div id="column-one">
<div id="p-cactions" class="portlet">
<h5>Views</h5>
<ul>
<li id="ca-nstab-main" class="selected"><a href="/research/wiki/index.php/REAP_Study_on_Vocabulary_Stretch_%28Spring_2006%29">Article</a></li>
<li id="ca-talk" class="new"><a href="/research/wiki/index.php?title=Talk:REAP_Study_on_Vocabulary_Stretch_%28Spring_2006%29&action=edit">Discussion</a></li>
<li id="ca-edit"><a href="/research/wiki/index.php?title=REAP_Study_on_Vocabulary_Stretch_%28Spring_2006%29&action=edit">Edit</a></li>
<li id="ca-history"><a href="/research/wiki/index.php?title=REAP_Study_on_Vocabulary_Stretch_%28Spring_2006%29&action=history">History</a></li>
<li id="ca-move"><a href="/research/wiki/index.php/Special:Movepage/REAP_Study_on_Vocabulary_Stretch_%28Spring_2006%29">Move</a></li>
<li id="ca-watch"><a href="/research/wiki/index.php?title=REAP_Study_on_Vocabulary_Stretch_%28Spring_2006%29&action=watch">Watch</a></li>
</ul>
</div>
<div class="portlet" id="p-personal">
<h5>Personal tools</h5>
<div class="pBody">
<ul>
<li id="pt-userpage"><a href="/research/wiki/index.php/User:Feeney" class="new">Feeney</a></li>
<li id="pt-mytalk"><a href="/research/wiki/index.php/User_talk:Feeney" class="new">My talk</a></li>
<li id="pt-preferences"><a href="/research/wiki/index.php/Special:Preferences">Preferences</a></li>
<li id="pt-watchlist"><a href="/research/wiki/index.php/Special:Watchlist">My watchlist</a></li>
<li id="pt-mycontris"><a href="/research/wiki/index.php/Special:Contributions/Feeney">My contributions</a></li>
<li id="pt-logout"><a href="/research/wiki/index.php?title=Special:Userlogout&returnto=REAP_Study_on_Vocabulary_Stretch_(Spring_2006)">Log out</a></li>
</ul>
</div>
</div>
<div class="portlet" id="p-logo">
<a style="background-image: url(/research/wiki/skins/pslc_logo.png);" href="/research/wiki/index.php/Main_Page" title="Main Page"></a>
</div>
<script type="text/javascript"> if (window.isMSIE55) fixalpha(); </script>
<div class='portlet' id='p-navigation'>
<h5>Navigation</h5>
<div class='pBody'>
<ul>
<li id="n-mainpage"><a href="/research/wiki/index.php/Main_Page">Main Page</a></li>
<li id="n-Root-Node"><a href="/research/wiki/index.php/Root_node">Root Node</a></li>
<li id="n-Glossary"><a href="/research/wiki/index.php/Category:Glossary">Glossary</a></li>
<li id="n-recentchanges"><a href="/research/wiki/index.php/Special:Recentchanges">Recent changes</a></li>
<li id="n-randompage"><a href="/research/wiki/index.php/Special:Random">Random page</a></li>
<li id="n-help"><a href="/research/wiki/index.php/Help:Contents">Help</a></li>
</ul>
</div>
</div>
<div class='portlet' id='p-clusters'>
<h5>clusters</h5>
<div class='pBody'>
<ul>
<li id="n-Coordinative-Learning"><a href="/research/wiki/index.php/Coordinative_Learning">Coordinative Learning</a></li>
<li id="n-Interactive-Communication"><a href="/research/wiki/index.php/Interactive_Communication">Interactive Communication</a></li>
<li id="n-Refinement-and-Fluency"><a href="/research/wiki/index.php/Refinement_and_Fluency">Refinement and Fluency</a></li>
</ul>
</div>
</div>
<div class='portlet' id='p-courses'>
<h5>courses</h5>
<div class='pBody'>
<ul>
<li id="n-Algebra"><a href="/research/wiki/index.php/Algebra">Algebra</a></li>
<li id="n-Geometry"><a href="/research/wiki/index.php/Geometry">Geometry</a></li>
<li id="n-Chinese"><a href="/research/wiki/index.php/Chinese">Chinese</a></li>
<li id="n-French"><a href="/research/wiki/index.php/French">French</a></li>
<li id="n-English"><a href="/research/wiki/index.php/English">English</a></li>
<li id="n-Chemistry"><a href="/research/wiki/index.php/Chemistry">Chemistry</a></li>
<li id="n-Physics"><a href="/research/wiki/index.php/Physics">Physics</a></li>
</ul>
</div>
</div>
<div class='portlet' id='p-datashop'>
<h5>datashop</h5>
<div class='pBody'>
<ul>
<li id="n-DataShop-Pipeline"><a href="/research/wiki/index.php/DataShop_Pipeline">DataShop Pipeline</a></li>
</ul>
</div>
</div>
<div id="p-search" class="portlet">
<h5><label for="searchInput">Search</label></h5>
<div id="searchBody" class="pBody">
<form action="/research/wiki/index.php/Special:Search" id="searchform"><div>
<input id="searchInput" name="search" type="text" accesskey="f" value="" />
<input type='submit' name="go" class="searchButton" id="searchGoButton" value="Go" /> 
<input type='submit' name="fulltext" class="searchButton" value="Search" />
</div></form>
</div>
</div>
<div class="portlet" id="p-tb">
<h5>Toolbox</h5>
<div class="pBody">
<ul>
<li id="t-whatlinkshere"><a href="/research/wiki/index.php/Special:Whatlinkshere/REAP_Study_on_Vocabulary_Stretch_%28Spring_2006%29">What links here</a></li>
<li id="t-recentchangeslinked"><a href="/research/wiki/index.php/Special:Recentchangeslinked/REAP_Study_on_Vocabulary_Stretch_%28Spring_2006%29">Related changes</a></li>
<li id="t-upload"><a href="/research/wiki/index.php/Special:Upload">Upload file</a></li>
<li id="t-specialpages"><a href="/research/wiki/index.php/Special:Specialpages">Special pages</a></li>
<li id="t-print"><a href="/research/wiki/index.php?title=REAP_Study_on_Vocabulary_Stretch_%28Spring_2006%29&printable=yes">Printable version</a></li> <li id="t-permalink"><a href="/research/wiki/index.php?title=REAP_Study_on_Vocabulary_Stretch_%28Spring_2006%29&oldid=4799">Permanent link</a></li> </ul>
</div>
</div>
</div>
<div class="visualClear"></div>
<div id="footer">
<div id="f-poweredbyico"><a href="http://www.mediawiki.org/"><img src="/research/wiki/skins/common/images/poweredby_mediawiki_88x31.png" alt="MediaWiki" /></a></div>
<ul id="f-list">
<li id="lastmod"> This page was last modified 15:26, 14 April 2007.</li>
<li id="viewcount">This page has been accessed 17 times.</li>
<li id="privacy"><a href="/research/wiki/index.php/Pslc:Privacy_policy" title="Pslc:Privacy policy">Privacy policy</a></li>
<li id="about"><a href="/research/wiki/index.php/Pslc:About" title="Pslc:About">About Pslc</a></li>
<li id="disclaimer"><a href="/research/wiki/index.php/Pslc:General_disclaimer" title="Pslc:General disclaimer">Disclaimers</a></li>
</ul>
</div>
<script type="text/javascript"> if (window.runOnloadHook) runOnloadHook();</script>
</div>

</body></html>

REAP Study on the Correlation Between Automatically and Manually Generated Reading Comprehension Questions (Summer 2007)

2007-07-25T16:59:51Z

Feeney:

REAP Study on the Correlation Between Automatically and Manually Generated Reading Comprehension Questions (Summer 2007)

2007-07-25T16:58:27Z

Feeney:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en" dir="ltr">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="keywords" content="REAP Study on the Correlation Between Automatically and Manually Generated Reading Comprehension Questions (Summer 2007)" />
<link rel="shortcut icon" href="/favicon.ico" />
<title> REAP Study on the Correlation Between Automatically and Manually Generated Reading Comprehension Questions (Summer 2007)- Pslc</title>
<style type="text/css" media="screen,projection">/*<![CDATA[*/ @import "/research/wiki/skins/monobook/main.css?7"; /*]]>*/</style>
<link rel="stylesheet" type="text/css" media="print" href="/research/wiki/skins/common/commonPrint.css" />





<script type="text/javascript">var skin = 'monobook';var stylepath = '/research/wiki/skins';</script>
<script type="text/javascript" src="/research/wiki/skins/common/wikibits.js"></script>
<script type="text/javascript" src="/research/wiki/index.php?title=-&action=raw&smaxage=0&gen=js"></script>
<style type="text/css">/*<![CDATA[*/
@import "/research/wiki/index.php?title=MediaWiki:Common.css&action=raw&ctype=text/css&smaxage=18000";
@import "/research/wiki/index.php?title=MediaWiki:Monobook.css&action=raw&ctype=text/css&smaxage=18000";
@import "/research/wiki/index.php?title=-&action=raw&gen=css&maxage=18000";
/*]]>*/</style>

</head>
<body class="ns-0 ltr">
<div id="globalWrapper">
<div id="column-content">
<div id="content">
<a name="top" id="top"></a>
<h1 class="firstHeading"> REAP Study on the Correlation Between Automatically and Manually Generated Reading Comprehension Questions (Summer 2007)</h1>
<div id="bodyContent">
<h3 id="siteSub">From Pslc</h3>
<div id="contentSub"></div>
<div id="jump-to-nav">Jump to: <a href="#column-one">navigation</a>, <a href="#searchInput">search</a></div> 
<table id="toc" class="toc" summary="Contents"><tr><td><div id="toctitle"><h2>Contents</h2></div>
<ul>
<li class='toclevel-1'><a href="#REAP_Study_on_Question_Types"><span class="tocnumber">1</span> <span class="toctext"> REAP Study on the Correlation Between Automatically and Manually Generated Reading Comprehension Questions (Summer 2007)</span></a>
<ul>
<li class='toclevel-2'><a href="#Logistical_Information"><span class="tocnumber">1.1</span> <span class="toctext">Logistical Information</span></a></li>
<li class='toclevel-2'><a href="#Abstract"><span class="tocnumber">1.2</span> <span class="toctext">Abstract</span></a></li>
<li class='toclevel-2'><a href="#Glossary"><span class="tocnumber">1.3</span> <span class="toctext">Glossary</span></a></li>
<li class='toclevel-2'><a href="#Research_question"><span class="tocnumber">1.4</span> <span class="toctext">Research question</span></a></li>
<li class='toclevel-2'><a href="#Dependent_variables"><span class="tocnumber">1.5</span> <span class="toctext">Dependent variables</span></a></li>
<li class='toclevel-2'><a href="#Independent_variables"><span class="tocnumber">1.6</span> <span class="toctext">Independent variables</span></a></li>
<li class='toclevel-2'><a href="#Hypotheses"><span class="tocnumber">1.7</span> <span class="toctext">Hypotheses</span></a></li>
<li class='toclevel-2'><a href="#Explanation"><span class="tocnumber">1.8</span> <span class="toctext">Explanation</span></a></li>
<li class='toclevel-2'><a href="#Notes"><span class="tocnumber">1.9</span> <span class="toctext">Notes</span></a></li>
<li class='toclevel-2'><a href="#Descendents"><span class="tocnumber">1.10</span> <span class="toctext">Descendents</span></a></li>
<li class='toclevel-2'><a href="#Annotated_bibliography"><span class="tocnumber">1.11</span> <span class="toctext">Annotated bibliography</span></a></li>
<p>
Curran, James R. Curran & Moens, Marc. (2002a). Improvements in automatic thesaurus extraction. Proceedings of the Workshop of the ACL Special Interest Group on the Lexicon (SIGLEX), 59-66.

Davoudi, Mohammad. (2005). Inference Generation Skill and Text Comprehension. The Reading Matrix, 5(1), 106-123.

Heilman, M., Collins-Thompson, K., Callan, J. & Eskenazi, M. (2006). Classroom success of an Intelligent Tutoring System for lexical practice and reading comprehension. Proceedingsof the Ninth International Conference on Spoken Language Processing.

Heilman, M. & Eskenazi, M. In Press. Application of Automatic Thesaurus Extraction for Computer Generation of Vocabulary Questions. Proceedings of the SLaTE Workshop on Speech and Language Technology in Education.

</p>
</ul>
</li>
</ul>
</td></tr></table><script type="text/javascript"> if (window.showTocToggle) { var tocShowText = "show"; var tocHideText = "hide"; showTocToggle(); } </script>
<div class="editsection" style="float:right;margin-left:5px;">[<a href="/research/wiki/index.php?title=REAP_Study_on_Question_Types_%28Summer_2006%29&action=edit&section=1" title="Edit section: REAP Study on the Correlation Between Automatically and Manually Generated Reading Comprehension Questions (Summer 2007)">edit</a>]</div><a name=" REAP_Study_on_the_Correlation_Between_Automatically_and_Manually_Generated_Reading_Comprehension_Questions_"></a><h2> REAP Study on the Correlation Between Automatically and Manually Generated Reading Comprehension Questions"></</h2>
<div class="editsection" style="float:right;margin-left:5px;">[<a href="/research/wiki/index.php?title= REAP_Study_on_the_Correlation_Between_Automatically_and_Manually_Generated_Reading_Comprehension_Questions_"></%28Summer_2007%29&action=edit&section=2" title="Edit section: Logistical Information">edit</a>]</div><a name="Logistical_Information"></a><h3> Logistical Information </h3>
<table border="1">
<caption>
</caption>
<tr>
<td> <b>Contributors</b> </td><td> Christine Feeney, Michael Heilman, </td></tr>
<tr>
<td> <b>Study Start Date</b> </td><td> June, 2007
</td></tr>
<tr>
<td> <b>Study End Date</b> </td><td> July, 2007
</td></tr>
<tr>
<td> <b>Learnlab Courses</b> </td><td> none </td></tr>
<tr>
<td> <b>Number of Students</b> </td><td> 30
</td></tr>
<tr>
<td> <b>Total Participant Hours (est.)</b> </td><td> 30
</td></tr>
<tr>
<td> <b>Data in Datashop</b> </td><td> no
</td></tr></table>
<p><br />
</p>
<div class="editsection" style="float:right;margin-left:5px;">[<a href="/research/wiki/index.php?title=REAP_Study_on_Question_Types_%28Summer_2006%29&action=edit&section=3" title="Edit section: Abstract">edit</a>]</div><a name="Abstract"></a><h3> Abstract </h3>
<p> Previous psychological research has identified two types of comprehension: shallow, in which people can reproduce information, and deep, in which people can comprehend the meaning of information. In addition, researchers have found that these two types of comprehension are separate and disparate. The PSLC currently employs an English as a Second Language vocabulary tutor, REAP, that uses shallow, automatically generated, comprehension questions to check that students are actively reading, rather than just skimming, practice reading passages. The purpose of the current study was to examine performance on the REAP-type questions with manually authored, deeper, reading comprehension questions. Participants were thirty undergraduate students (male = 11) participating in summer research programs at Carnegie Mellon University. The researcher predicted a positive correlation between the two testing types, which was supported by the data (r = .366, p < 0.0005, one-tailed t-test).
</p>
<div class="editsection" style="float:right;margin-left:5px;">[<a href="/research/wiki/index.php?title=REAP_Study_on_Question_Types_%28Summer_2006%29&action=edit&section=4" title="Edit section: Glossary">edit</a>]</div><a name="Glossary"></a><h3> Glossary </h3>
<p><i>Shallow Comprehension:</i> Processing directed towards reproducing the learning material without necessarily understanding it.
</p><p><i>Deep Comprehension:</i> Processing directed towards comprehending the intended meaning of the learning material. </p>
<div class="editsection" style="float:right;margin-left:5px;">[<a href="/research/wiki/index.php?title=REAP_Study_on_Question_Types_%28Summer_2006%29&action=edit&section=5" title="Edit section: Research question">edit</a>]</div><a name="Research_question"></a><h3> Research question </h3>
<p>Does a relationship exist between the two question types?
</p>
<div class="editsection" style="float:right;margin-left:5px;">[<a href="/research/wiki/index.php?title= REAP_Study_on_the_Correlation_Between_Automatically_and_Manually_Generated_Reading_Comprehension_Questions_%28Summer_2007%29&action=edit&section=6" title="Edit section: Dependent variables">edit</a>]</div><a name="Dependent_variables"></a><h3> Dependent variables </h3>
<p>Percent correct on each question type.
</p>
<div class="editsection" style="float:right;margin-left:5px;">[<a href="/research/wiki/index.php?title= REAP_Study_on_the_Correlation_Between_Automatically_and_Manually_Generated_Reading_Comprehension_Questions_%28Summer_2007%29&action=edit&section=7" title="Edit section: Independent variables">edit</a>]</div><a name="Independent_variables"></a><h3> Independent variables </h3>
<p>Chosen Reading Passages and question types.
</p>
<div class="editsection" style="float:right;margin-left:5px;">[<a href="/research/wiki/index.php?title= REAP_Study_on_the_Correlation_Between_Automatically_and_Manually_Generated_Reading_Comprehension_Questions_%28Summer_2007%29&action=edit&section=8" title="Edit section: Hypotheses">edit</a>]</div><a name="Hypotheses"></a><h3> Hypotheses </h3>
<p> The researchers hypothesized that there would be a significant positive correlation between performance on the two question types.
<p>
<div class="editsection" style="float:right;margin-left:5px;">[<a href="/research/wiki/index.php?title= REAP_Study_on_the_Correlation_Between_Automatically_and_Manually_Generated_Reading_Comprehension_Questions_%28Summer_2007%29&action=edit&section=9" title="Edit section: Explanation">edit</a>]</div><a name="Explanation"></a><h3> Explanation </h3>
<p> The researcher hypothesized a strong positive correlation between performance on shallow and deep reading comprehension questions, which was partially supported by the data analysis. When looking at the participants’ percentages of correct responses for all the questions, these two variables had a medium positive correlation of .366 (p < 0.0005, one-tailed test). This indicates that there is a relationship between the two variables, which can be expressed by the regression equation: Percent Correct of Deep Questions = 0.09 + 0.51 * Percent Correct of Shallow Questions (Figure 2). The R2 value of 0.13 means that overall, thirteen percent of the variance of performance in answering deep comprehension questions can be explained by some mechanism that is at work in answering shallow comprehension questions.
</p>
<div class="editsection" style="float:right;margin-left:5px;">[<a href="/research/wiki/index.php?title=REAP_Study_on_Question_Types_%28Summer_2006%29&action=edit&section=10" title="Edit section: Notes">edit</a>]</div><a name="Notes"></a><h3> Notes </h3>
<p>
</p>
<div class="editsection" style="float:right;margin-left:5px;">[<a href="/research/wiki/index.php?title=REAP_Study_on_Question_Types_%28Summer_2006%29&action=edit&section=11" title="Edit section: Descendents">edit</a>]</div><a name="Descendents"></a><h3> Descendents </h3>
<div class="editsection" style="float:right;margin-left:5px;">[<a href="/research/wiki/index.php?title=REAP_Study_on_Question_Types_%28Summer_2006%29&action=edit&section=12" title="Edit section: Annotated bibliography">edit</a>]</div><a name="Annotated_bibliography"></a><h3> Annotated bibliography </h3>
<div class="printfooter">
Retrieved from "<a href="http://www.learnlab.org/research/wiki/index.php/REAP_Study_on_Question_Types_%28Summer_2006%29">http://www.learnlab.org/research/wiki/index.php/REAP_Study_on_Question_Types_%28Summer_2006%29</a>"</div>

<div class="visualClear"></div>
</div>
</div>
</div>
<div id="column-one">
<div id="p-cactions" class="portlet">
<h5>Views</h5>
<ul>
<li id="ca-nstab-main" class="selected"><a href="/research/wiki/index.php/REAP_Study_on_Question_Types_%28Summer_2006%29">Article</a></li>
<li id="ca-talk" class="new"><a href="/research/wiki/index.php?title=Talk:REAP_Study_on_Question_Types_%28Summer_2006%29&action=edit">Discussion</a></li>
<li id="ca-edit"><a href="/research/wiki/index.php?title=REAP_Study_on_Question_Types_%28Summer_2006%29&action=edit">Edit</a></li>
<li id="ca-history"><a href="/research/wiki/index.php?title=REAP_Study_on_Question_Types_%28Summer_2006%29&action=history">History</a></li>
<li id="ca-move"><a href="/research/wiki/index.php/Special:Movepage/REAP_Study_on_Question_Types_%28Summer_2006%29">Move</a></li>
<li id="ca-watch"><a href="/research/wiki/index.php?title=REAP_Study_on_Question_Types_%28Summer_2006%29&action=watch">Watch</a></li>
</ul>
</div>
<div class="portlet" id="p-personal">
<h5>Personal tools</h5>
<div class="pBody">
<ul>
<li id="pt-userpage"><a href="/research/wiki/index.php/User:Feeney" class="new">Feeney</a></li>
<li id="pt-mytalk"><a href="/research/wiki/index.php/User_talk:Feeney" class="new">My talk</a></li>
<li id="pt-preferences"><a href="/research/wiki/index.php/Special:Preferences">Preferences</a></li>
<li id="pt-watchlist"><a href="/research/wiki/index.php/Special:Watchlist">My watchlist</a></li>
<li id="pt-mycontris"><a href="/research/wiki/index.php/Special:Contributions/Feeney">My contributions</a></li>
<li id="pt-logout"><a href="/research/wiki/index.php?title=Special:Userlogout&returnto=REAP_Study_on_Question_Types_(Summer_2006)">Log out</a></li>
</ul>
</div>
</div>
<div class="portlet" id="p-logo">
<a style="background-image: url(/research/wiki/skins/pslc_logo.png);" href="/research/wiki/index.php/Main_Page" title="Main Page"></a>
</div>
<script type="text/javascript"> if (window.isMSIE55) fixalpha(); </script>
<div class='portlet' id='p-navigation'>
<h5>Navigation</h5>
<div class='pBody'>
<ul>
<li id="n-mainpage"><a href="/research/wiki/index.php/Main_Page">Main Page</a></li>
<li id="n-Root-Node"><a href="/research/wiki/index.php/Root_node">Root Node</a></li>
<li id="n-Glossary"><a href="/research/wiki/index.php/Category:Glossary">Glossary</a></li>
<li id="n-recentchanges"><a href="/research/wiki/index.php/Special:Recentchanges">Recent changes</a></li>
<li id="n-randompage"><a href="/research/wiki/index.php/Special:Random">Random page</a></li>
<li id="n-help"><a href="/research/wiki/index.php/Help:Contents">Help</a></li>
</ul>
</div>
</div>
<div class='portlet' id='p-clusters'>
<h5>clusters</h5>
<div class='pBody'>
<ul>
<li id="n-Coordinative-Learning"><a href="/research/wiki/index.php/Coordinative_Learning">Coordinative Learning</a></li>
<li id="n-Interactive-Communication"><a href="/research/wiki/index.php/Interactive_Communication">Interactive Communication</a></li>
<li id="n-Refinement-and-Fluency"><a href="/research/wiki/index.php/Refinement_and_Fluency">Refinement and Fluency</a></li>
</ul>
</div>
</div>
<div class='portlet' id='p-courses'>
<h5>courses</h5>
<div class='pBody'>
<ul>
<li id="n-Algebra"><a href="/research/wiki/index.php/Algebra">Algebra</a></li>
<li id="n-Geomtery"><a href="/research/wiki/index.php/Geometry">Geomtery</a></li>
<li id="n-Chinese"><a href="/research/wiki/index.php/Chinese">Chinese</a></li>
<li id="n-French"><a href="/research/wiki/index.php/French">French</a></li>
<li id="n-English"><a href="/research/wiki/index.php/English">English</a></li>
<li id="n-Chemistry"><a href="/research/wiki/index.php/Chemistry">Chemistry</a></li>
<li id="n-Physics"><a href="/research/wiki/index.php/Physics">Physics</a></li>
</ul>
</div>
</div>
<div class='portlet' id='p-datashop'>
<h5>datashop</h5>
<div class='pBody'>
<ul>
<li id="n-DataShop-Pipeline"><a href="/research/wiki/index.php/DataShop_Pipeline">DataShop Pipeline</a></li>
</ul>
</div>
</div>
<div id="p-search" class="portlet">
<h5><label for="searchInput">Search</label></h5>
<div id="searchBody" class="pBody">
<form action="/research/wiki/index.php/Special:Search" id="searchform"><div>
<input id="searchInput" name="search" type="text" accesskey="f" value="" />
<input type='submit' name="go" class="searchButton" id="searchGoButton" value="Go" /> 
<input type='submit' name="fulltext" class="searchButton" value="Search" />
</div></form>
</div>
</div>
<div class="portlet" id="p-tb">
<h5>Toolbox</h5>
<div class="pBody">
<ul>
<li id="t-whatlinkshere"><a href="/research/wiki/index.php/Special:Whatlinkshere/REAP_Study_on_Question_Types_%28Summer_2006%29">What links here</a></li>
<li id="t-recentchangeslinked"><a href="/research/wiki/index.php/Special:Recentchangeslinked/REAP_Study_on_Question_Types_%28Summer_2006%29">Related changes</a></li>
<li id="t-upload"><a href="/research/wiki/index.php/Special:Upload">Upload file</a></li>
<li id="t-specialpages"><a href="/research/wiki/index.php/Special:Specialpages">Special pages</a></li>
<li id="t-print"><a href="/research/wiki/index.php?title=REAP_Study_on_Question_Types_%28Summer_2006%29&printable=yes">Printable version</a></li> <li id="t-permalink"><a href="/research/wiki/index.php?title=REAP_Study_on_Question_Types_%28Summer_2006%29&oldid=4798">Permanent link</a></li> </ul>
</div>
</div>
</div>
<div class="visualClear"></div>
<div id="footer">
<div id="f-poweredbyico"><a href="http://www.mediawiki.org/"><img src="/research/wiki/skins/common/images/poweredby_mediawiki_88x31.png" alt="MediaWiki" /></a></div>
<ul id="f-list">
<li id="lastmod"> This page was last modified 15:24, 14 April 2007.</li>
<li id="viewcount">This page has been accessed 17 times.</li>
<li id="privacy"><a href="/research/wiki/index.php/Pslc:Privacy_policy" title="Pslc:Privacy policy">Privacy policy</a></li>
<li id="about"><a href="/research/wiki/index.php/Pslc:About" title="Pslc:About">About Pslc</a></li>
<li id="disclaimer"><a href="/research/wiki/index.php/Pslc:General_disclaimer" title="Pslc:General disclaimer">Disclaimers</a></li>
</ul>
</div>
<script type="text/javascript"> if (window.runOnloadHook) runOnloadHook();</script>
</div>

</body></html>

REAP main

2007-07-24T00:20:28Z

Feeney: /* Descendents */

== The REAP Project Root Node ==

=== Logistical Information ===

See individual studies under Descendants.

=== Abstract ===
The REAP project investigates the effects of implicit and explicit instruction on the learning of word meanings. The REAP tutor ([[link|http://reap.cs.cmu.edu]]) is in LearnLab studies at the English Language Institute at the Univ. of Pittsburgh.

=== Glossary ===

''Implicit Learning'': The learning of meaning for a word from the context in which that word occurs.

''Explicit Learning'': The learning of meaning for a word from explicit instruction, either in the form of a dictionary definition or a practice exercise.

''Intentional Learning'': The learning of meanings for words that are focused on in instructional materials. Targeted words might be highlighted in a reading, or practiced in vocabulary exercises.

''Incidental Learning'': The learning of meanings for words that appear in instructional materials but are not explicitly taught. For example, a student may learn an unknown word that occurs in a practice reading but is not the focus of the reading.

=== Research question ===

The general question of the [[Coordinative Learning]] cluster that the REAP project is aimed at is the following: "When and how should explicit explanations be added or requested of students before, during, or after example study and problem solving practice?" In REAP's case, the verbal explication comes primarily in the form of dictionary definitions of vocabulary words. However, in contrast to other domains, explicit instruction is the first means of instruction for vocabulary rather than the secondary means of instruction. In the case of vocabulary for second language learners, explicit instruction may provide the most economical means of learning, but this learning may be shallow. The question for language learning is then, "How much and in what ways does adding or requesting coordinated implicit examples (of word usage in context) increase robust learning?"

Additional questions:
What are the characteristics of instruction that lead to robust knowledge of vocabulary? Do students learn better from explicit instruction (e.g., dictionary definitions), or can they learn implicitly (e.g., from context)? To what extent does incidental learning of vocabulary occur (e.g., learning of words not targeted by a practice reading, but looked up by the student as he or she read the text.)?

'''Examples of REAP Issues in a table of Different Types of Instruction and Learning'''

{| border="1"
|+
|-
| ||'''Passive''' || '''Active''' || '''Interactive'''
|-
| '''Explicit (general)''' || Dictionary definitions || || Multiple-choice definition questions (for practice or assessment)
|-
| '''Implicit (instance)''' || Interpreting meaning in context while reading || Novel sentence production (post-test measure of transfer) || Cloze Vocabulary Practice Questions
|}

=== Dependent variables ===
[[Normal post-test]] scores, on exercises similar to practice exercises. These test items are usually cloze/fill-in-the-blank questions on target vocabulary words.

[[Long-term retention]] test scores similar to post-test but administered at least one month later.

[[Transfer]] of knowledge: practice vocabulary questions of other types, sentence production tasks for target words, correct use of words in writing assignments for other courses.

=== Independent variables ===
e.g., availability of dictionary definitions, presentation of definitions, highlighting of target words in readings, personalization of readings by topics of interest, etc.

=== Hypotheses ===
Students require both explicit and implicit instruction to effectively acquire vocabulary. Students do not efficiently learn vocabulary implicitly (e.g., from context alone). Similarly, learning solely from explicit dictionary definitions will lead to shallow acquisition. Students who only have definitions available will have difficulty generalizing their knowledge and producing sentences using new vocabulary. Therefore, words need to be seen in context with definitions available in order for robust learning to occur.

=== Explanation ===

=== Notes ===

The words which the REAP tutor provides practice for are from the Academic Word List (Coxhead, 2000).

Beginning with the Spring 2006 study, dictionary definitions were available for all words in a reading unless noted otherwise. Definitions came from the Cambridge Advanced Learner's Dictionary (dictionary.cambridge.org) unless noted otherwise.

ELI students use REAP in their reading class about 40 minutes per week as part of a 4 hour reading course. The rest of the class time is spent on the textbook and other vocabulary exercises.

=== Descendents ===
*REAP Summer 2005 Study

*REAP Fall 2005 Study

*[[REAP Study on Vocabulary Stretch (Spring 2006)]].

*[[REAP Study on Question Types (Summer 2006)]].

*[[REAP Study on Personalization of Readings by Topic (Fall 2006)]].

*[[REAP Comparison to Classroom Instruction (Fall 2006)]].

*[[REAP Study on Focusing of Attention (Spring 2007)]].

*[[ELI Study on Activity Theory and student use of REAP (Spring 2007)]].

*[[REAP Study on Word Sense Disambiguation (Summer 2007)]].

*[[REAP Study on Hearing Words Pronounced (Summer 2007)]].

*[[REAP Study on the Correlation Between Automatically and Manually Generated Reading Comprehension Questions (Summer 2007)]].

=== Annotated bibliography ===

Craik, F. & Lockhardt, R. (1972). Levels of processing: a framework for memory research. Journal of Verbal Learning and Verbal Behavior, 11, 671-684.

Cobb, T. (2006). The old vocabulary, the new vocabulary, and the Arabic learner. A TESOL symposium on Vocabulary words matter: The importance of vocabulary in English Language Teaching and Learning. Alexandria, VA: TESOL.

Coxhead, Averil (2000) A New Academic Word List. TESOL Quarterly, 34(2): 213-238.

Ellis, N. C., & Sinclair, S. (1996). Working memory in the acquisition of vocabulary and syntax. Quarterly journal of experimental psychology, 49, 234-250

Hatch, E., & Brown, C. (1995). Vocabulary, semantics, and language education. Cambridge: Cambridge University Press.

Hulstijn, J., Hollander, M., & Greidanus, T. (1996). Incidental vocabulary learning by advanced foreign language students: the influence of marginal glosses, dictionary use, and re-occurrence of unknown words. Modern Language Journal, 80, 327-339.

Krashen, S. (1989). We acquire vocabulary and spelling by reading: additional evidence for the input hypothesis. Modern Language Journal, 73, 440-464.

McCarthy, M. (1994). A new look at vocabulary in EFL. Applied Linguistics, 5, 12-22.

Meara, P. (1984). The study of lexis in interlanguage. In A. Davies, C. Criper & A. Howatt (Eds.), Interlanguage (pp. 225-235).

Nation, I. S. P. (2001). Learning vocabulary in another language. Cambridge: Cambridge University Press.

Prince, P. (1996). Second language vocabulary learning: the role of context versus translations as a function of proficiency. Modern Language Journal, 80, 478-493.

Richards, J. (1976). The role of vocabulary teaching. TESOL Quarterly, 10, 77-89.

Stanowicz, K. E. (1986). Matthew effects in reading: some consequences of individual differences in the aquisition of literacy. Reading Research Quarterly, 21, 360-407.

[http://reap.cs.cmu.edu/ REAP Web Page]

Heilman, M., Collins-Thompson, K., Callan, J. & Eskanazi, M. (2006). Classroom success of an Intelligent Tutoring System for lexical practice and reading comprehension. Proceedings of the Ninth International Conference on Spoken Language Processing. [http://reap.cs.cmu.edu/Papers/REAP-interspeech06-final.pdf [PDF]]

Heilman, M. & Eskenazi, M. (2006). Language Learning: Challenges for Intelligent Tutoring Systems. Proceedings of the Workshop of Intelligent Tutoring Systems for Ill-Defined Domains. 8th International Conference on Intelligent Tutoring Systems. [Heilman, M. & Eskenazi, M. (2006). Language Learning: Challenges for Intelligent Tutoring Systems. Proceedings of the Workshop of Intelligent Tutoring Systems for Ill-Defined Domains. 8th International Conference on Intelligent Tutoring Systems. [http://reap.cs.cmu.edu/Papers/ITS06_illdefinedworkshop_HeilmanEskenazi.pdf [PDF]]

J. Brown, G. Frishkoff, and M. Eskenazi. (2005). "Automatic question generation for vocabulary assessment." In Proceedings of HLT/EMNLP 2005. Vancouver, B.C. [http://reap.cs.cmu.edu/Papers/hlt05-jonbrown.pdf [PDF]]

J. Brown and M. Eskenazi. (2005). "Student, text and curriculum modeling for reader-specific document retrieval." In Proceedings of the IASTED International Conference on Human-Computer Interaction 2005. Phoenix, AZ. [http://reap.cs.cmu.edu/Papers/IASTED-HCI-05-jonbrown.pdf [PDF]]

J. Brown and M. Eskenazi. (2004.) "Retrieval of authentic documents for reader-specific lexical practice." In Proceedings of InSTIL/ICALL Symposium 2004. Venice, Italy. [http://reap.cs.cmu.edu/Papers/InSTIL04-jonbrown.pdf PDF]]

A. Juffs, L. Wilson, M. Eskenazi, J. Callan, J. Brown, K. Collins-Thompson, M. Heilman, T. Pelletreau, and J. Sanders. (2006.) "Robust learning of vocabulary: investigating the relationship between learner behaviour and the acquisition of vocabulary" (poster). At The 40th Annual TESOL Convention and Exhibit (TESOL 2006). [http://reap.cs.cmu.edu/Papers/TESOL-REAPPoster03-06.pdf [PDF]]

K. Collins-Thompson and J. Callan. (2004.) "A language modeling approach to predicting reading difficulty." In Proceedings of the HLT/NAACL 2004 Conference. Boston. [http://www.cs.cmu.edu/~callan/Papers/hlt04-kct.pdf [PDF]]

Pelletreau, T. (2006). Computer assisted vocabulary acquisition in the ESL classroom. Unpublished Master of Arts, University of Pittsburgh, Pittsburgh. [http://etd.library.pitt.edu/ETD/available/etd-08082006-035505/]