Theory Wiki - User contributions [en]

REAP main

2009-11-22T17:44:53Z

Gparent:

== The REAP Project Root Node ==

=== Logistical Information ===

See individual studies under Descendants.

=== Abstract ===
The REAP project investigates the effects of implicit and explicit instruction on the learning of word meanings. The REAP tutor ([[link|http://reap.cs.cmu.edu]]) is in LearnLab studies at the English Language Institute at the Univ. of Pittsburgh.

=== Glossary ===

''Implicit Learning'': The learning of meaning for a word from the context in which that word occurs.

''Explicit Learning'': The learning of meaning for a word from explicit instruction, either in the form of a dictionary definition or a practice exercise.

''Intentional Learning'': The learning of meanings for words that are focused on in instructional materials. Targeted words might be highlighted in a reading, or practiced in vocabulary exercises.

''Incidental Learning'': The learning of meanings for words that appear in instructional materials but are not explicitly taught. For example, a student may learn an unknown word that occurs in a practice reading but is not the focus of the reading.

=== Research question ===

The general question of the [[Coordinative Learning]] cluster that the REAP project is aimed at is the following: "When and how should explicit explanations be added or requested of students before, during, or after example study and problem solving practice?" In REAP's case, the verbal explication comes primarily in the form of dictionary definitions of vocabulary words. However, in contrast to other domains, explicit instruction is the first means of instruction for vocabulary rather than the secondary means of instruction. In the case of vocabulary for second language learners, explicit instruction may provide the most economical means of learning, but this learning may be shallow. The question for language learning is then, "How much and in what ways does adding or requesting coordinated implicit examples (of word usage in context) increase robust learning?"

Additional questions:
What are the characteristics of instruction that lead to robust knowledge of vocabulary? Do students learn better from explicit instruction (e.g., dictionary definitions), or can they learn implicitly (e.g., from context)? To what extent does incidental learning of vocabulary occur (e.g., learning of words not targeted by a practice reading, but looked up by the student as he or she read the text.)?

'''Examples of REAP Issues in a table of Different Types of Instruction and Learning'''

{| border="1"
|+
|-
| ||'''Passive''' || '''Active''' || '''Interactive'''
|-
| '''Explicit (general)''' || Dictionary definitions || || Multiple-choice definition questions (for practice or assessment)
|-
| '''Implicit (instance)''' || Interpreting meaning in context while reading || Novel sentence production (post-test measure of transfer) || Cloze Vocabulary Practice Questions
|}

=== Dependent variables ===
[[Normal post-test]] scores, on exercises similar to practice exercises. These test items are usually cloze/fill-in-the-blank questions on target vocabulary words.

[[Long-term retention]] test scores similar to post-test but administered at least one month later.

[[Transfer]] of knowledge: practice vocabulary questions of other types, sentence production tasks for target words, correct use of words in writing assignments for other courses.

=== Independent variables ===

* Availability of dictionary definitions (control condition: not able to click on target words to get a definition)

[[Image:dictionaryDefinitionSmall.jpg|Dictionary Definition]]

* Highlighting of target words in readings (control condition: target words not hightlighted)

* Personalization of interest (control condition: document retrieval algorithm does not take into account students' preferences)

[[Image:interest.jpg|Personalization of interest]]

=== Hypotheses ===
Students require both explicit and implicit instruction to effectively acquire vocabulary. Students do not efficiently learn vocabulary implicitly (e.g., from context alone). Similarly, learning solely from explicit dictionary definitions will lead to shallow acquisition. Students who only have definitions available will have difficulty generalizing their knowledge and producing sentences using new vocabulary. Therefore, words need to be seen in context with definitions available in order for robust learning to occur.

=== Explanation ===

=== Notes ===

The words which the REAP tutor provides practice for are from the Academic Word List (Coxhead, 2000).

Beginning with the Spring 2006 study, dictionary definitions were available for all words in a reading unless noted otherwise. Definitions came from the Cambridge Advanced Learner's Dictionary (dictionary.cambridge.org) unless noted otherwise.

ELI students use REAP in their reading class about 40 minutes per week as part of a 4 hour reading course. The rest of the class time is spent on the textbook and other vocabulary exercises.

=== Descendents ===

*[[REAP Study on Vocabulary Stretch (Spring 2006)]].

*[[REAP Study on Question Types (Summer 2006)]].

*[[REAP Study on Personalization of Readings by Topic (Fall 2006)]].

*[[REAP Comparison to Classroom Instruction (Fall 2006)]].

*[[REAP Study on Focusing of Attention (Spring 2007)]].

*[[ELI Study on Activity Theory and student use of REAP (Spring 2007)]].

*[[REAP Study on Word Sense Disambiguation (Summer 2007)]].

*[[REAP Study on Hearing Words Pronounced (Summer 2007)]].

*[[REAP Study on Highlighting Target Vocabulary to Focus Learner Attention]]

*[[REAP_Multimodal_Learning_Fall_09 | REAP Study on Multi-Modal Learning (Fall 2009)]].

=== Annotated bibliography ===

Craik, F. & Lockhardt, R. (1972). Levels of processing: a framework for memory research. Journal of Verbal Learning and Verbal Behavior, 11, 671-684.

Cobb, T. (2006). The old vocabulary, the new vocabulary, and the Arabic learner. A TESOL symposium on Vocabulary words matter: The importance of vocabulary in English Language Teaching and Learning. Alexandria, VA: TESOL.

Coxhead, Averil (2000) A New Academic Word List. TESOL Quarterly, 34(2): 213-238.

Ellis, N. C., & Sinclair, S. (1996). Working memory in the acquisition of vocabulary and syntax. Quarterly journal of experimental psychology, 49, 234-250

Hatch, E., & Brown, C. (1995). Vocabulary, semantics, and language education. Cambridge: Cambridge University Press.

Hulstijn, J., Hollander, M., & Greidanus, T. (1996). Incidental vocabulary learning by advanced foreign language students: the influence of marginal glosses, dictionary use, and re-occurrence of unknown words. Modern Language Journal, 80, 327-339.

Krashen, S. (1989). We acquire vocabulary and spelling by reading: additional evidence for the input hypothesis. Modern Language Journal, 73, 440-464.

McCarthy, M. (1994). A new look at vocabulary in EFL. Applied Linguistics, 5, 12-22.

Meara, P. (1984). The study of lexis in interlanguage. In A. Davies, C. Criper & A. Howatt (Eds.), Interlanguage (pp. 225-235).

Nation, I. S. P. (2001). Learning vocabulary in another language. Cambridge: Cambridge University Press.

Prince, P. (1996). Second language vocabulary learning: the role of context versus translations as a function of proficiency. Modern Language Journal, 80, 478-493.

Richards, J. (1976). The role of vocabulary teaching. TESOL Quarterly, 10, 77-89.

Stanowicz, K. E. (1986). Matthew effects in reading: some consequences of individual differences in the aquisition of literacy. Reading Research Quarterly, 21, 360-407.

[http://reap.cs.cmu.edu/ REAP Web Page]

Heilman, M., Collins-Thompson, K., Callan, J. & Eskanazi, M. (2006). Classroom success of an Intelligent Tutoring System for lexical practice and reading comprehension. Proceedings of the Ninth International Conference on Spoken Language Processing. [http://reap.cs.cmu.edu/Papers/REAP-interspeech06-final.pdf [PDF]]

Heilman, M. & Eskenazi, M. (2006). Language Learning: Challenges for Intelligent Tutoring Systems. Proceedings of the Workshop of Intelligent Tutoring Systems for Ill-Defined Domains. 8th International Conference on Intelligent Tutoring Systems. [Heilman, M. & Eskenazi, M. (2006). Language Learning: Challenges for Intelligent Tutoring Systems. Proceedings of the Workshop of Intelligent Tutoring Systems for Ill-Defined Domains. 8th International Conference on Intelligent Tutoring Systems. [http://reap.cs.cmu.edu/Papers/ITS06_illdefinedworkshop_HeilmanEskenazi.pdf [PDF]]

J. Brown, G. Frishkoff, and M. Eskenazi. (2005). "Automatic question generation for vocabulary assessment." In Proceedings of HLT/EMNLP 2005. Vancouver, B.C. [http://reap.cs.cmu.edu/Papers/hlt05-jonbrown.pdf [PDF]]

J. Brown and M. Eskenazi. (2005). "Student, text and curriculum modeling for reader-specific document retrieval." In Proceedings of the IASTED International Conference on Human-Computer Interaction 2005. Phoenix, AZ. [http://reap.cs.cmu.edu/Papers/IASTED-HCI-05-jonbrown.pdf [PDF]]

J. Brown and M. Eskenazi. (2004.) "Retrieval of authentic documents for reader-specific lexical practice." In Proceedings of InSTIL/ICALL Symposium 2004. Venice, Italy. [http://reap.cs.cmu.edu/Papers/InSTIL04-jonbrown.pdf PDF]]

A. Juffs, L. Wilson, M. Eskenazi, J. Callan, J. Brown, K. Collins-Thompson, M. Heilman, T. Pelletreau, and J. Sanders. (2006.) "Robust learning of vocabulary: investigating the relationship between learner behaviour and the acquisition of vocabulary" (poster). At The 40th Annual TESOL Convention and Exhibit (TESOL 2006). [http://reap.cs.cmu.edu/Papers/TESOL-REAPPoster03-06.pdf [PDF]]

K. Collins-Thompson and J. Callan. (2004.) "A language modeling approach to predicting reading difficulty." In Proceedings of the HLT/NAACL 2004 Conference. Boston. [http://www.cs.cmu.edu/~callan/Papers/hlt04-kct.pdf [PDF]]

Pelletreau, T. (2006). Computer assisted vocabulary acquisition in the ESL classroom. Unpublished Master of Arts, University of Pittsburgh, Pittsburgh. [http://etd.library.pitt.edu/ETD/available/etd-08082006-035505/]

The following is a literature review of CALL and vocabulary instruction. It was written by Ben Friedline after discussion with Alan Juffs about what kinds of improvements we would like to see in REAP as it develops for full classroom use.

CALL and Vocabulary Acquisition:
Implications for the Design of CALL for the ESL Classroom

Benjamin E. Friedline
University of Pittsburgh
Department of Linguistics

Dr. Alan Juffs

April 9, 2008

CALL and Vocabulary Acquisition:
Implications for the Design of CALL for the ESL Classroom
Abstract
The purpose of this paper is to provide suggestions on how to design materials for English as a Second Language (ESL) CALL-based vocabulary instruction based on current research findings. To this end, the current paper examines: 1) how CALL can be used to promote long-term vocabulary retention, 2) the issue of word selection in CALL design, 3) how output related computer-mediated tasks can influence vocabulary retention, and 4) the idea of individual differences in students’ approaches to CALL. The purpose of this paper is to review the findings from the literature on CALL and vocabulary acquisition not to propose new research, but to provide a principled foundation for the design of ESL CALL-based vocabulary instruction that promotes learning and long-term retention of vocabulary words.

Introduction
Current vocabulary acquisition research highlights an L2 learner’s need to acquire a large working vocabulary in order to comprehend and communicate effectively in an L2 (Folse, 2006; Nation, 2002). Research has also revealed that the frequency of occurrence of vocabulary words in English drops considerably after the first 3,000 most frequent words (Nation, 2002). Three overarching research agendas have resulted from these findings. Firstly, researchers are concerned with determining which words and the quantity of words students need to learn in order to ‘succeed’ in an academic program in English (Coxhead, 2001; Hunt & Beglar, 2002; Nation, 2002). Secondly, many research studies have been concerned with the question of how to teach vocabulary to ESL students in the most effective way. Some research studies have shown that extensive reading leads to enhanced vocabulary learning and long-term retention (Grabe, 2002; Horst, 2005; Renandya & Jacobs, 2002) and how different exercise types can influence vocabulary retention (Folse, 2006). Additionally, other research has highlighted the importance of interaction (output) in getting learners to retain the word knowledge that they have gained through intensive reading (Hunt & Beglar, 2002; Nation, 2002). Lastly, an important trend in past research (not-necessarily vocabulary research) has been to determine how individual differences contribute to what learners are capable of learning from completing a particular task (Dörnyei & Skehan, 2000).
The previously cited research has all been conducted within face-to-face contexts. Interestingly enough, many of the same issues that are present in research in face-to-face conversations and vocabulary research are also issues within the literature that looks at CALL and vocabulary acquisition. The purpose of this paper is to review the findings from the literature on CALL and vocabulary acquisition not to propose new research, but to provide a principled foundation for the design of ESL CALL-based vocabulary instruction that promotes learning and long-term retention of vocabulary words. Additionally, this paper assumes that users of the CALL applications will be academically oriented and needing to learn vocabulary in order to engage in there respective academic programs. This paper is important for developers of ESL curriculums and materials. For the materials developer, this paper provides a theoretical rationale for designing CALL applications that promote vocabulary acquisition. On the other hand, this paper provides a ‘checklist’ for curriculum developers who are looking to implement CALL programs into their language learning curriculum. That is, if certain applications do not meet the criteria that are proposed in this paper, then one should be cautious in bringing such a program into his/her instructional context.
Word Presentation in CALL
Learners and language instructors will find that CALL offers a variety of methods for presenting new vocabulary words. In spite of the plethora of options that CALL programs can offer in terms of activity design, research reveals that the selection of words and the way that they are presented will directly impact vocabulary learning. Firstly, I will examine the issue of which words to teach through CALL. Based on past research on word selection for vocabulary instruction within face-to-face conversation (Coxhead, 2001; Nation, 2002), it makes sense to conclude that CALL applications should also present words that are not frequent in the input and words that will be needed to engage in academic disciplines. Indeed, the literature suggests that some CALL applications (Juffs, Friedline, Wilson, Eskenazi, & Heilman, submitted) are being designed to teach words from word lists such as the AWL (Coxhead, 2001).
Unfortunately, these programs are the exception to the general trend in the design of CALL programs that promote vocabulary acquisition. The words in these programs are generally selected based on native speaker judgments (Groot, 2000), high-frequency (Tozcu & Coady, 2006), or from an in-class textbook (Allum, 2004). The problem with the vocabulary used in these programs pertains to the question: Do learners need these words? In other words, there is no theoretical basis for the selection of the words that are being taught by these CALL applications. This argument can also be applied to words that are excluded from CALL applications. For example, Nesselhauf and Tschichold (2002) reviewed seven different CALL vocabulary-building programs and found that collocations were largely neglected in these programs. This result is surprising if we consider the frequency and importance of multi-word units in everyday speech and writing (Nesselhauf & Tschichold, 2002), and the difficulty that second-language learners have in learning them (Chan & Liou, 2005; Sun & Wang, 2003).
A recent approach that has been forwarded to address the need to teach collocations is the use of online concordancers. Benefits of the use of concordancers may include contextual word learning (Cobb, 1999), extensive exposure to naturally occurring input (Chan & Liou, 2005; Sun & Wang, 2003), and consciousness raising of the existence of multi-word units among L2 learners (Chan & Liou, 2005). Cobb (1999) found that learners who used a concordancer learned definitional knowledge of words and were able to transfer the knowledge of words to the comprehension of novel texts, both long-term and short-term. On the other hand, students in Cobb’s (1999) study who used a list and dictionary method (instead of a concordancer) to learn new words were able to learn definitional meanings, but these meanings were not retained in the long-term, nor were the learners able to transfer their knowledge to novel texts. Chan and Liou (2005) explored the learning of collocations through a web-based bilingual concordancer by using a one-group pretest/posttest experiment. The results of their study revealed that the use of a bilingual concordancer could aid learners in acquiring immediate collocational knowledge; however, this knowledge was not retained over time.
The previously cited research highlights some of the problems with the words that have been selected for instructional purposes by current CALL vocabulary-building software. These problems include the lack of a theoretical basis for word selection and the exclusion of multi-word units such as collocations. In light of these problems, a synthesis of current research may provide insights for improving CALL vocabulary-building software. Firstly, the words that students learn should be selected based upon their frequency (e.g., AWL) and their utility for functioning within an academic discipline. Additionally, CALL vocabulary-building software should focus on teaching multi-word units because of the difficulties that such words present for second language learners. Current research (e.g., Cobb, 1999) suggests that concordancers may help learners to acquire collocational knowledge by presenting them with large amounts of contextual evidence.
CALL and Vocabulary Retention
Recent CALL research suggests that CALL vocabulary tutors can promote long-term vocabulary retention. Grace (1998) did a study in which she used a pretest/posttest/delayed posttest design to test to see if L1 translations of target words helped learners to learn and retain vocabulary words. There were 181 beginning-level French students involved in this study; 89 were placed in the experimental condition (L1 Translations) and 92 were placed in the control condition (no L1 Translations). Grace found that both groups exhibited gains in short-term and long-term vocabulary retention, but the experimental condition demonstrated more significant gains in vocabulary retention. Grace attributes the increased retention for the experimental condition to the fact that having an L1 translation allows a learner to process a word more deeply by connecting an L2 word with and L1 word than learners who have to guess the meaning of a word from context. Additionally, Grace mentions that having learners guess word meaning from context can be detrimental to vocabulary learning because learners will often make a wrong guess and store an incorrect word meaning to memory (Grace, 1998, p. 540).
Groot (2000) conducted a study in which he compared the performance of two groups of students on two different vocabulary learning tasks. In this study, one of the groups learned words from a bilingual list, while the other group learned words from a CALL program that modeled the vocabulary acquisition process (e.g., noticing, storage, and consolidation of storage). On the immediate posttest and delayed posttest, the control group (bilingual list) outperformed the experimental group (CALL) in terms of vocabulary knowledge. However, in one of the two repeated experiments, the researchers observed less retention loss for the experimental condition on the delayed posttest. The researcher attributes the enhanced retention to the CALL application’s ability to promote deep-processing of the lexical items and proposes that the best method is an approach that combines bilingual lists and CALL. These findings are supported by other studies that have specifically examined the benefits that L1 translations (Grace, 1998) and language-specific feedback that promotes depth of processing (Zapata & Sagarra, 2007) have on vocabulary retention.
Also related to depth of processing, a study by Zapata and Sagarra (2007) compared the performance of two groups of students using different vocabulary exercises for 24 weeks; 245 students worked with an online workbook, and 304 students used a paper workbook. The two different types of workbooks contained exactly the same content. During the first 2-4 months, there was no significant difference between the two groups. However, the students using the online workbook performed better than the control group on tests of lexical knowledge 6-8 months after the treatment. The researchers suggest that this difference was due to the differences in feedback between the two types of activities. That is, CALL was able to provide learners with immediate feedback, whereas students had to wait on their teachers to receive feedback on the written workbook.
The final study that will be included in this section on CALL and vocabulary acquisition is a study by Nakata (2008) in which the researcher compared the effectiveness of vocabulary instruction through computers, cards, and lists in terms of vocabulary retention. The computer was designed to accommodate optimal spaced learning, which takes place when the item to be learned is rehearsed over a long period of time in a sequence that promotes deep processing. (Nakata, 2008, p. 5-6). In this study, 226 first and second year high-school EFL learners were placed into one of three groups: 1) a list group, 2) a card group, and 3) a computer group. The results on the immediate posttest revealed that there was no significant difference between the three groups in terms of vocabulary knowledge. However, a significant difference was found between the scores on the delayed posttest, which was administered 4 days after the treatment. According to these results, the PC group scored significantly higher than the list group, but there was no significant difference between the card group and the computer group. These results suggest that learning vocabulary from cards and CALL are more effective for long-term vocabulary retention, especially when one considers the idea of optimal spaced learning that comes from cognitive psychology. In this case, cards and computer work better than lists because they are not subject to rehearsal in a static order.
In sum, CALL tutors should be designed to promote long-term retention of vocabulary items. As the previously cited research indicates, there are several different interrelated factors that can contribute to long-term retention. That is, CALL has the ability to promote deep-processing of lexical items through immediate feedback, optimally spaced rehearsal, and by presenting L2 learners with the option of consulting an L1 translation of a vocabulary item.
Negotiated Interaction and Vocabulary Retention
Research suggests that vocabulary retention is enhanced if learners perform an output related task in connection with a vocabulary learning task (Hunt & Beglar, 2002; Nation, 2002). This same principle holds true for vocabulary tasks that take place through CALL. Smith (2004) conducted a study of NNS-NNS interactions over Internet-Relay Chat (IRC) in order to determine if NNS-NNS interactions through Computer-Mediated Communication (CMC) could facilitate learners’ abilities to recognize and produce new lexical items, and if the effects would hold up over time. His study included 45 intermediate-level ESL students, who interacted over IRC to negotiate the meanings of several target words over 5 sessions. According to Smith’s (2004) study, negotiated items were learned much better than non-negotiated items, and the vocabulary gains were shown to hold up over time. Thus, this study provides support for Long’s interaction hypothesis (as cited in Mitchell & Myles, 2004) as it relates to CALL and vocabulary acquisition.
A study by De la Fuente (2003) also provides support for the utility of computer mediated interaction in lexical development. In her study, De la Fuente (2003) researched if computer mediated interactions could help students to advance their lexical L2 interlanguage. The participants in this study included 24 students from 3 different classes of second-semester elementary Spanish. The researcher divided students into pairs and placed them in either the face-to-face interaction group or the computer mediated group and had them negotiate the meaning of 43 Spanish nouns through an information gap activity over 2 days. In terms of receptive and productive oral acquisition, both groups showed gains in vocabulary knowledge; however, the face-to-face group outperformed the computer group on both measures. In terms of receptive and productive written acquisition, both groups showed gains, but this time neither group’s performance was significantly different on either of these measures. Once again, this study provides support for the fact that computer mediated interaction can lead to lexical development. The superior performance on oral tasks by the face-to-face group would be expected since they received oral input and quite possibly textual input from their interlocutors, whereas the computer group was limited to text-based input only.
In sum, the previously cited research provides evidence that interactive CMC tasks enhance vocabulary learning. Thus, CALL designers may want to consider including interactive tasks (e.g., online chat or voice chat) that involve learner-to-learner interactions as opposed to interactions that involve a learner ‘interacting’ with a computer interface.
Individual Differences and CALL
The final portion of this literature review will discuss how the idea of individual differences might affect the design of CALL vocabulary tutors. In a study of a CALL vocabulary tutor in an Intensive English Program (IEP), Juffs et al. (submitted) found that the goals of the students and researchers were not aligned. That is, the researchers expected students to engage in extensive reading and to have some interest in the articles; however, some learners spent too much time reading a single article and looking up every word, while others sped through the articles as fast as they could just to get done with the articles. This study is important because it shows that CALL design involves ‘getting into the heads of the learners’ and being able to anticipate how they will use the program. Certainly, this is no easy task.
Current research includes some ideas for designing CALL to accommodate many individual differences. I say ‘many’ here because I know that it is never possible to please every learner. Firstly, research highlights the importance of having a user-friendly and contemporary-looking interface (Nesselhauf & Tschichold, 2002). Although factors such as a programs’ looks may not appear to be all that important, learners believe otherwise, and will often choose a program based on how it looks as opposed to the program’s true merit. Secondly, Tozcu and Coady (2004) mention that learners in their study can individualize their own word lists while using the New Lexis software. This was not the main goal of the article, but the idea of allowing users to manipulate and individualize some aspects of their vocabulary-learning experience seems to be a good feature to add to any CALL program to accommodate individual differences. Lastly, a CALL vocabulary tutor (especially one that promotes vocabulary acquisition through extensive reading) should allow learners to select texts based on interest (Juffs et al., submitted; Renandya & Jacobs, 2002). Accommodating student interest is important for any extensive reading program because interesting readings can provide students with the necessary intrinsic motivation to read large amounts of texts.
Past research on the use of CALL vocabulary tutors has revealed the importance of considering individual differences when designing CALL applications. To some extent, it is impossible to predict exactly how students will use a CALL application, but it is important to pilot test new tutors and to interview students who use the program to understand students’ attitudes towards the program and predict problems that students might have. This research also indicates that a tutor’s looks, ease of use, and ability to be appropriated for a learner’s individual use are important factors to consider for designing CALL vocabulary tutors. Furthermore, CALL should allow learners to choose texts that are of interest to individual learners.
Conclusions: Best Practices for Designing CALL Vocabulary Tutors
In light of the previously cited research, some general conclusions can be drawn in regards to ‘best practices’ for the design of vocabulary tutors. Firstly, CALL vocabulary tutors should teach words that students need to learn in order to enter into academic programs and make them noticeable for language learners. Words that come from the AWL or another well-known word list should be taught, as well as frequent multi-word units such as phrasal verbs and collocations. Current research has shown that concordancers or bilingual concordancers may be useful in promoting knowledge and recognition of multi-word units. Secondly, CALL tutors should: 1) provide immediate feedback to learners, 2) allow optimally spaced rehearsal, and 3) provide learners with L1 translations so that learners can verify the meanings of words in their L1 before committing them to memory. It is argued that all of these factors lead to deep-processing, which is linked to long-term retention. Thirdly, CALL vocabulary tutors should include features which allow learners to interact with other learners while or immediately after using CALL. Lastly, CALL designers must consider the role of individual differences in vocabulary learning from CALL. In order to accommodate individual differences, CALL programs should be easy to use, modern-looking, and customizable, while at the same time providing learners with interesting materials to learn from.

References
Allum, P. (2004). Evaluation of CALL: Initial vocabulary learning. ReCALL, 16(2), 488-501.
Chan, T., & Liou, H. (2005). Effects of web-based concordancing instruction on EFL students' learning of verb-noun collocations. Computer Assisted Language Learning, 18(3), 231-250.
Cobb, T. (1999). Breadth and depth of lexical acquisition with hands-on concordancing. Computer Assisted Language Learning, 12(4), 345-360.
Coxhead, A. (2001) A new academic word list. TESOL Quarterly 34: 213-238.
De la Fuente, M. (2003). Is SLA interactionist theory relevant to CALL? A study on the effects of computer-mediated interaction in L2 vocabulary acquisition. Computer Assisted Language Learning, 16(1), 47-81.
Dörnyei, Z., & Skehan, P. (2005). Individual differences in second language learning. In C. Doughty & M. Long (Eds.), The handbook of second language acquisition. Massachusetts: Blackwell.
Folse, K. (2006). The effect of type of written exercise on L2 vocabulary retention. TESOL Quarterly, 40(2), 273-293.
Grabe, W. (2002). Dilemmas for the development of second language reading abilities. In J. C. Richards & W. A. Renandya (Eds.), Methodology in language teaching. New York: Cambridge University Press.
Grace, C. A. (1998). Retention of word meanings inferred from context and sentence-level translations: Implications for the design of beginning-level CALL software. The Modern Language Journal, 82(iv), 533-544.

Groot, P. (2000). Computer assisted second language vocabulary acquisition. Language Learning and Technology, 4, 60-81.
Horst, M. (2005). Learning L2 vocabulary through extensive reading: A measurement study. The Canadian Modern Language Review, 61(3), 355-382.
Hunt, A., & Beglar, D. (2002). Current research and practice in teaching vocabulary. In J. C. Richards & W. A. Renandya (Eds.), Methodology in language teaching. New York: Cambridge University Press.
Juffs, A., Friedline, B., Wilson, L., Eskenazi, M., & Heilman, M. (submitted). Activity theory and computer-assisted learning of English vocabulary. Applied Linguistics.
Mitchell, R., & Myles, F. (2004). Second language learning theories (2nd ed.). New York: Hodder Arnold.
Nakata, T. (2008). English vocabulary learning with word lists, word cards and computers: Implications from cognitive psychology research for optimal spaced learning. ReCALL, 20(1), 3-20.
Nation, P. (2002). Best practice in vocabulary teaching and learning. In J. C. Richards & W. A. Renandya (Eds.), Methodology in language teaching. New York: Cambridge University Press.
Nesselhauf, N., & Tschichold, C. (2002). Collocations in CALL: An investigation of vocabulary-building software for EFL. Computer Assisted Language Learning, 15(3), 251-279.
Renandya, W., & Jacobs, G. (2002). Extensive reading: Why aren't we all doing it? In J. C. Richards & W. A. Renandya (Eds.), Methodology in language teaching. New York: Cambridge University Press.

Smith, B. (2004). Computer-mediated negotiated interaction and lexical acquisition. SSLA, 26, 365-398.
Sun, Y., & Wang, L. (2003). Concordancers in the EFL classroom: Cognitive approaches and collocation difficulty. Computer Assisted Language Learning, 16(1), 83-94.
Tozcu, A., & Coady, J. (2004). Successful learning of frequent vocabulary through CALL also benefits reading comprehension speed. Computer Assisted Language Learning, 17(5), 473-495.
Zapata, G., & Sagarra, N. (2007). CALL on hold: The delayed benefits of an online workbook on L2 vocabulary learning. Computer Assisted Language Learning, 20(2), 153-171.

REAP main

2009-11-22T17:43:47Z

Gparent:

== The REAP Project Root Node ==

=== Logistical Information ===

See individual studies under Descendants.

=== Abstract ===
The REAP project investigates the effects of implicit and explicit instruction on the learning of word meanings. The REAP tutor ([[link|http://reap.cs.cmu.edu]]) is in LearnLab studies at the English Language Institute at the Univ. of Pittsburgh.

=== Glossary ===

''Implicit Learning'': The learning of meaning for a word from the context in which that word occurs.

''Explicit Learning'': The learning of meaning for a word from explicit instruction, either in the form of a dictionary definition or a practice exercise.

''Intentional Learning'': The learning of meanings for words that are focused on in instructional materials. Targeted words might be highlighted in a reading, or practiced in vocabulary exercises.

''Incidental Learning'': The learning of meanings for words that appear in instructional materials but are not explicitly taught. For example, a student may learn an unknown word that occurs in a practice reading but is not the focus of the reading.

=== Research question ===

The general question of the [[Coordinative Learning]] cluster that the REAP project is aimed at is the following: "When and how should explicit explanations be added or requested of students before, during, or after example study and problem solving practice?" In REAP's case, the verbal explication comes primarily in the form of dictionary definitions of vocabulary words. However, in contrast to other domains, explicit instruction is the first means of instruction for vocabulary rather than the secondary means of instruction. In the case of vocabulary for second language learners, explicit instruction may provide the most economical means of learning, but this learning may be shallow. The question for language learning is then, "How much and in what ways does adding or requesting coordinated implicit examples (of word usage in context) increase robust learning?"

Additional questions:
What are the characteristics of instruction that lead to robust knowledge of vocabulary? Do students learn better from explicit instruction (e.g., dictionary definitions), or can they learn implicitly (e.g., from context)? To what extent does incidental learning of vocabulary occur (e.g., learning of words not targeted by a practice reading, but looked up by the student as he or she read the text.)?

'''Examples of REAP Issues in a table of Different Types of Instruction and Learning'''

{| border="1"
|+
|-
| ||'''Passive''' || '''Active''' || '''Interactive'''
|-
| '''Explicit (general)''' || Dictionary definitions || || Multiple-choice definition questions (for practice or assessment)
|-
| '''Implicit (instance)''' || Interpreting meaning in context while reading || Novel sentence production (post-test measure of transfer) || Cloze Vocabulary Practice Questions
|}

=== Dependent variables ===
[[Normal post-test]] scores, on exercises similar to practice exercises. These test items are usually cloze/fill-in-the-blank questions on target vocabulary words.

[[Long-term retention]] test scores similar to post-test but administered at least one month later.

[[Transfer]] of knowledge: practice vocabulary questions of other types, sentence production tasks for target words, correct use of words in writing assignments for other courses.

=== Independent variables ===

* Availability of dictionary definitions (control condition: not able to click on target words to get a definition)

[[Image:dictionaryDefinitionSmall.jpg|Dictionary Definition]]

* Highlighting of target words in readings (control condition: target words not hightlighted)

* Personalization of interest (control condition: document retrieval algorithm does not take into account students' preferences)

[[Image:interest.jpg|Personalization of interest]]

=== Hypotheses ===
Students require both explicit and implicit instruction to effectively acquire vocabulary. Students do not efficiently learn vocabulary implicitly (e.g., from context alone). Similarly, learning solely from explicit dictionary definitions will lead to shallow acquisition. Students who only have definitions available will have difficulty generalizing their knowledge and producing sentences using new vocabulary. Therefore, words need to be seen in context with definitions available in order for robust learning to occur.

=== Explanation ===

=== Notes ===

The words which the REAP tutor provides practice for are from the Academic Word List (Coxhead, 2000).

Beginning with the Spring 2006 study, dictionary definitions were available for all words in a reading unless noted otherwise. Definitions came from the Cambridge Advanced Learner's Dictionary (dictionary.cambridge.org) unless noted otherwise.

ELI students use REAP in their reading class about 40 minutes per week as part of a 4 hour reading course. The rest of the class time is spent on the textbook and other vocabulary exercises.

=== Descendents ===

*[[REAP Study on Vocabulary Stretch (Spring 2006)]].

*[[REAP Study on Question Types (Summer 2006)]].

*[[REAP Study on Personalization of Readings by Topic (Fall 2006)]].

*[[REAP Comparison to Classroom Instruction (Fall 2006)]].

*[[REAP Study on Focusing of Attention (Spring 2007)]].

*[[ELI Study on Activity Theory and student use of REAP (Spring 2007)]].

*[[REAP Study on Word Sense Disambiguation (Summer 2007)]].

*[[REAP Study on Hearing Words Pronounced (Summer 2007)]].

*[[REAP Study on Highlighting Target Vocabulary to Focus Learner Attention]]

*[[REAP_Multimodal_Learning | REAP Study on Multi-Modal Learning (Fall 2009)]].

=== Annotated bibliography ===

Craik, F. & Lockhardt, R. (1972). Levels of processing: a framework for memory research. Journal of Verbal Learning and Verbal Behavior, 11, 671-684.

Cobb, T. (2006). The old vocabulary, the new vocabulary, and the Arabic learner. A TESOL symposium on Vocabulary words matter: The importance of vocabulary in English Language Teaching and Learning. Alexandria, VA: TESOL.

Coxhead, Averil (2000) A New Academic Word List. TESOL Quarterly, 34(2): 213-238.

Ellis, N. C., & Sinclair, S. (1996). Working memory in the acquisition of vocabulary and syntax. Quarterly journal of experimental psychology, 49, 234-250

Hatch, E., & Brown, C. (1995). Vocabulary, semantics, and language education. Cambridge: Cambridge University Press.

Hulstijn, J., Hollander, M., & Greidanus, T. (1996). Incidental vocabulary learning by advanced foreign language students: the influence of marginal glosses, dictionary use, and re-occurrence of unknown words. Modern Language Journal, 80, 327-339.

Krashen, S. (1989). We acquire vocabulary and spelling by reading: additional evidence for the input hypothesis. Modern Language Journal, 73, 440-464.

McCarthy, M. (1994). A new look at vocabulary in EFL. Applied Linguistics, 5, 12-22.

Meara, P. (1984). The study of lexis in interlanguage. In A. Davies, C. Criper & A. Howatt (Eds.), Interlanguage (pp. 225-235).

Nation, I. S. P. (2001). Learning vocabulary in another language. Cambridge: Cambridge University Press.

Prince, P. (1996). Second language vocabulary learning: the role of context versus translations as a function of proficiency. Modern Language Journal, 80, 478-493.

Richards, J. (1976). The role of vocabulary teaching. TESOL Quarterly, 10, 77-89.

Stanowicz, K. E. (1986). Matthew effects in reading: some consequences of individual differences in the aquisition of literacy. Reading Research Quarterly, 21, 360-407.

[http://reap.cs.cmu.edu/ REAP Web Page]

Heilman, M., Collins-Thompson, K., Callan, J. & Eskanazi, M. (2006). Classroom success of an Intelligent Tutoring System for lexical practice and reading comprehension. Proceedings of the Ninth International Conference on Spoken Language Processing. [http://reap.cs.cmu.edu/Papers/REAP-interspeech06-final.pdf [PDF]]

Heilman, M. & Eskenazi, M. (2006). Language Learning: Challenges for Intelligent Tutoring Systems. Proceedings of the Workshop of Intelligent Tutoring Systems for Ill-Defined Domains. 8th International Conference on Intelligent Tutoring Systems. [Heilman, M. & Eskenazi, M. (2006). Language Learning: Challenges for Intelligent Tutoring Systems. Proceedings of the Workshop of Intelligent Tutoring Systems for Ill-Defined Domains. 8th International Conference on Intelligent Tutoring Systems. [http://reap.cs.cmu.edu/Papers/ITS06_illdefinedworkshop_HeilmanEskenazi.pdf [PDF]]

J. Brown, G. Frishkoff, and M. Eskenazi. (2005). "Automatic question generation for vocabulary assessment." In Proceedings of HLT/EMNLP 2005. Vancouver, B.C. [http://reap.cs.cmu.edu/Papers/hlt05-jonbrown.pdf [PDF]]

J. Brown and M. Eskenazi. (2005). "Student, text and curriculum modeling for reader-specific document retrieval." In Proceedings of the IASTED International Conference on Human-Computer Interaction 2005. Phoenix, AZ. [http://reap.cs.cmu.edu/Papers/IASTED-HCI-05-jonbrown.pdf [PDF]]

J. Brown and M. Eskenazi. (2004.) "Retrieval of authentic documents for reader-specific lexical practice." In Proceedings of InSTIL/ICALL Symposium 2004. Venice, Italy. [http://reap.cs.cmu.edu/Papers/InSTIL04-jonbrown.pdf PDF]]

A. Juffs, L. Wilson, M. Eskenazi, J. Callan, J. Brown, K. Collins-Thompson, M. Heilman, T. Pelletreau, and J. Sanders. (2006.) "Robust learning of vocabulary: investigating the relationship between learner behaviour and the acquisition of vocabulary" (poster). At The 40th Annual TESOL Convention and Exhibit (TESOL 2006). [http://reap.cs.cmu.edu/Papers/TESOL-REAPPoster03-06.pdf [PDF]]

K. Collins-Thompson and J. Callan. (2004.) "A language modeling approach to predicting reading difficulty." In Proceedings of the HLT/NAACL 2004 Conference. Boston. [http://www.cs.cmu.edu/~callan/Papers/hlt04-kct.pdf [PDF]]

Pelletreau, T. (2006). Computer assisted vocabulary acquisition in the ESL classroom. Unpublished Master of Arts, University of Pittsburgh, Pittsburgh. [http://etd.library.pitt.edu/ETD/available/etd-08082006-035505/]

The following is a literature review of CALL and vocabulary instruction. It was written by Ben Friedline after discussion with Alan Juffs about what kinds of improvements we would like to see in REAP as it develops for full classroom use.

CALL and Vocabulary Acquisition:
Implications for the Design of CALL for the ESL Classroom

Benjamin E. Friedline
University of Pittsburgh
Department of Linguistics

Dr. Alan Juffs

April 9, 2008

CALL and Vocabulary Acquisition:
Implications for the Design of CALL for the ESL Classroom
Abstract
The purpose of this paper is to provide suggestions on how to design materials for English as a Second Language (ESL) CALL-based vocabulary instruction based on current research findings. To this end, the current paper examines: 1) how CALL can be used to promote long-term vocabulary retention, 2) the issue of word selection in CALL design, 3) how output related computer-mediated tasks can influence vocabulary retention, and 4) the idea of individual differences in students’ approaches to CALL. The purpose of this paper is to review the findings from the literature on CALL and vocabulary acquisition not to propose new research, but to provide a principled foundation for the design of ESL CALL-based vocabulary instruction that promotes learning and long-term retention of vocabulary words.

Introduction
Current vocabulary acquisition research highlights an L2 learner’s need to acquire a large working vocabulary in order to comprehend and communicate effectively in an L2 (Folse, 2006; Nation, 2002). Research has also revealed that the frequency of occurrence of vocabulary words in English drops considerably after the first 3,000 most frequent words (Nation, 2002). Three overarching research agendas have resulted from these findings. Firstly, researchers are concerned with determining which words and the quantity of words students need to learn in order to ‘succeed’ in an academic program in English (Coxhead, 2001; Hunt & Beglar, 2002; Nation, 2002). Secondly, many research studies have been concerned with the question of how to teach vocabulary to ESL students in the most effective way. Some research studies have shown that extensive reading leads to enhanced vocabulary learning and long-term retention (Grabe, 2002; Horst, 2005; Renandya & Jacobs, 2002) and how different exercise types can influence vocabulary retention (Folse, 2006). Additionally, other research has highlighted the importance of interaction (output) in getting learners to retain the word knowledge that they have gained through intensive reading (Hunt & Beglar, 2002; Nation, 2002). Lastly, an important trend in past research (not-necessarily vocabulary research) has been to determine how individual differences contribute to what learners are capable of learning from completing a particular task (Dörnyei & Skehan, 2000).
The previously cited research has all been conducted within face-to-face contexts. Interestingly enough, many of the same issues that are present in research in face-to-face conversations and vocabulary research are also issues within the literature that looks at CALL and vocabulary acquisition. The purpose of this paper is to review the findings from the literature on CALL and vocabulary acquisition not to propose new research, but to provide a principled foundation for the design of ESL CALL-based vocabulary instruction that promotes learning and long-term retention of vocabulary words. Additionally, this paper assumes that users of the CALL applications will be academically oriented and needing to learn vocabulary in order to engage in there respective academic programs. This paper is important for developers of ESL curriculums and materials. For the materials developer, this paper provides a theoretical rationale for designing CALL applications that promote vocabulary acquisition. On the other hand, this paper provides a ‘checklist’ for curriculum developers who are looking to implement CALL programs into their language learning curriculum. That is, if certain applications do not meet the criteria that are proposed in this paper, then one should be cautious in bringing such a program into his/her instructional context.
Word Presentation in CALL
Learners and language instructors will find that CALL offers a variety of methods for presenting new vocabulary words. In spite of the plethora of options that CALL programs can offer in terms of activity design, research reveals that the selection of words and the way that they are presented will directly impact vocabulary learning. Firstly, I will examine the issue of which words to teach through CALL. Based on past research on word selection for vocabulary instruction within face-to-face conversation (Coxhead, 2001; Nation, 2002), it makes sense to conclude that CALL applications should also present words that are not frequent in the input and words that will be needed to engage in academic disciplines. Indeed, the literature suggests that some CALL applications (Juffs, Friedline, Wilson, Eskenazi, & Heilman, submitted) are being designed to teach words from word lists such as the AWL (Coxhead, 2001).
Unfortunately, these programs are the exception to the general trend in the design of CALL programs that promote vocabulary acquisition. The words in these programs are generally selected based on native speaker judgments (Groot, 2000), high-frequency (Tozcu & Coady, 2006), or from an in-class textbook (Allum, 2004). The problem with the vocabulary used in these programs pertains to the question: Do learners need these words? In other words, there is no theoretical basis for the selection of the words that are being taught by these CALL applications. This argument can also be applied to words that are excluded from CALL applications. For example, Nesselhauf and Tschichold (2002) reviewed seven different CALL vocabulary-building programs and found that collocations were largely neglected in these programs. This result is surprising if we consider the frequency and importance of multi-word units in everyday speech and writing (Nesselhauf & Tschichold, 2002), and the difficulty that second-language learners have in learning them (Chan & Liou, 2005; Sun & Wang, 2003).
A recent approach that has been forwarded to address the need to teach collocations is the use of online concordancers. Benefits of the use of concordancers may include contextual word learning (Cobb, 1999), extensive exposure to naturally occurring input (Chan & Liou, 2005; Sun & Wang, 2003), and consciousness raising of the existence of multi-word units among L2 learners (Chan & Liou, 2005). Cobb (1999) found that learners who used a concordancer learned definitional knowledge of words and were able to transfer the knowledge of words to the comprehension of novel texts, both long-term and short-term. On the other hand, students in Cobb’s (1999) study who used a list and dictionary method (instead of a concordancer) to learn new words were able to learn definitional meanings, but these meanings were not retained in the long-term, nor were the learners able to transfer their knowledge to novel texts. Chan and Liou (2005) explored the learning of collocations through a web-based bilingual concordancer by using a one-group pretest/posttest experiment. The results of their study revealed that the use of a bilingual concordancer could aid learners in acquiring immediate collocational knowledge; however, this knowledge was not retained over time.
The previously cited research highlights some of the problems with the words that have been selected for instructional purposes by current CALL vocabulary-building software. These problems include the lack of a theoretical basis for word selection and the exclusion of multi-word units such as collocations. In light of these problems, a synthesis of current research may provide insights for improving CALL vocabulary-building software. Firstly, the words that students learn should be selected based upon their frequency (e.g., AWL) and their utility for functioning within an academic discipline. Additionally, CALL vocabulary-building software should focus on teaching multi-word units because of the difficulties that such words present for second language learners. Current research (e.g., Cobb, 1999) suggests that concordancers may help learners to acquire collocational knowledge by presenting them with large amounts of contextual evidence.
CALL and Vocabulary Retention
Recent CALL research suggests that CALL vocabulary tutors can promote long-term vocabulary retention. Grace (1998) did a study in which she used a pretest/posttest/delayed posttest design to test to see if L1 translations of target words helped learners to learn and retain vocabulary words. There were 181 beginning-level French students involved in this study; 89 were placed in the experimental condition (L1 Translations) and 92 were placed in the control condition (no L1 Translations). Grace found that both groups exhibited gains in short-term and long-term vocabulary retention, but the experimental condition demonstrated more significant gains in vocabulary retention. Grace attributes the increased retention for the experimental condition to the fact that having an L1 translation allows a learner to process a word more deeply by connecting an L2 word with and L1 word than learners who have to guess the meaning of a word from context. Additionally, Grace mentions that having learners guess word meaning from context can be detrimental to vocabulary learning because learners will often make a wrong guess and store an incorrect word meaning to memory (Grace, 1998, p. 540).
Groot (2000) conducted a study in which he compared the performance of two groups of students on two different vocabulary learning tasks. In this study, one of the groups learned words from a bilingual list, while the other group learned words from a CALL program that modeled the vocabulary acquisition process (e.g., noticing, storage, and consolidation of storage). On the immediate posttest and delayed posttest, the control group (bilingual list) outperformed the experimental group (CALL) in terms of vocabulary knowledge. However, in one of the two repeated experiments, the researchers observed less retention loss for the experimental condition on the delayed posttest. The researcher attributes the enhanced retention to the CALL application’s ability to promote deep-processing of the lexical items and proposes that the best method is an approach that combines bilingual lists and CALL. These findings are supported by other studies that have specifically examined the benefits that L1 translations (Grace, 1998) and language-specific feedback that promotes depth of processing (Zapata & Sagarra, 2007) have on vocabulary retention.
Also related to depth of processing, a study by Zapata and Sagarra (2007) compared the performance of two groups of students using different vocabulary exercises for 24 weeks; 245 students worked with an online workbook, and 304 students used a paper workbook. The two different types of workbooks contained exactly the same content. During the first 2-4 months, there was no significant difference between the two groups. However, the students using the online workbook performed better than the control group on tests of lexical knowledge 6-8 months after the treatment. The researchers suggest that this difference was due to the differences in feedback between the two types of activities. That is, CALL was able to provide learners with immediate feedback, whereas students had to wait on their teachers to receive feedback on the written workbook.
The final study that will be included in this section on CALL and vocabulary acquisition is a study by Nakata (2008) in which the researcher compared the effectiveness of vocabulary instruction through computers, cards, and lists in terms of vocabulary retention. The computer was designed to accommodate optimal spaced learning, which takes place when the item to be learned is rehearsed over a long period of time in a sequence that promotes deep processing. (Nakata, 2008, p. 5-6). In this study, 226 first and second year high-school EFL learners were placed into one of three groups: 1) a list group, 2) a card group, and 3) a computer group. The results on the immediate posttest revealed that there was no significant difference between the three groups in terms of vocabulary knowledge. However, a significant difference was found between the scores on the delayed posttest, which was administered 4 days after the treatment. According to these results, the PC group scored significantly higher than the list group, but there was no significant difference between the card group and the computer group. These results suggest that learning vocabulary from cards and CALL are more effective for long-term vocabulary retention, especially when one considers the idea of optimal spaced learning that comes from cognitive psychology. In this case, cards and computer work better than lists because they are not subject to rehearsal in a static order.
In sum, CALL tutors should be designed to promote long-term retention of vocabulary items. As the previously cited research indicates, there are several different interrelated factors that can contribute to long-term retention. That is, CALL has the ability to promote deep-processing of lexical items through immediate feedback, optimally spaced rehearsal, and by presenting L2 learners with the option of consulting an L1 translation of a vocabulary item.
Negotiated Interaction and Vocabulary Retention
Research suggests that vocabulary retention is enhanced if learners perform an output related task in connection with a vocabulary learning task (Hunt & Beglar, 2002; Nation, 2002). This same principle holds true for vocabulary tasks that take place through CALL. Smith (2004) conducted a study of NNS-NNS interactions over Internet-Relay Chat (IRC) in order to determine if NNS-NNS interactions through Computer-Mediated Communication (CMC) could facilitate learners’ abilities to recognize and produce new lexical items, and if the effects would hold up over time. His study included 45 intermediate-level ESL students, who interacted over IRC to negotiate the meanings of several target words over 5 sessions. According to Smith’s (2004) study, negotiated items were learned much better than non-negotiated items, and the vocabulary gains were shown to hold up over time. Thus, this study provides support for Long’s interaction hypothesis (as cited in Mitchell & Myles, 2004) as it relates to CALL and vocabulary acquisition.
A study by De la Fuente (2003) also provides support for the utility of computer mediated interaction in lexical development. In her study, De la Fuente (2003) researched if computer mediated interactions could help students to advance their lexical L2 interlanguage. The participants in this study included 24 students from 3 different classes of second-semester elementary Spanish. The researcher divided students into pairs and placed them in either the face-to-face interaction group or the computer mediated group and had them negotiate the meaning of 43 Spanish nouns through an information gap activity over 2 days. In terms of receptive and productive oral acquisition, both groups showed gains in vocabulary knowledge; however, the face-to-face group outperformed the computer group on both measures. In terms of receptive and productive written acquisition, both groups showed gains, but this time neither group’s performance was significantly different on either of these measures. Once again, this study provides support for the fact that computer mediated interaction can lead to lexical development. The superior performance on oral tasks by the face-to-face group would be expected since they received oral input and quite possibly textual input from their interlocutors, whereas the computer group was limited to text-based input only.
In sum, the previously cited research provides evidence that interactive CMC tasks enhance vocabulary learning. Thus, CALL designers may want to consider including interactive tasks (e.g., online chat or voice chat) that involve learner-to-learner interactions as opposed to interactions that involve a learner ‘interacting’ with a computer interface.
Individual Differences and CALL
The final portion of this literature review will discuss how the idea of individual differences might affect the design of CALL vocabulary tutors. In a study of a CALL vocabulary tutor in an Intensive English Program (IEP), Juffs et al. (submitted) found that the goals of the students and researchers were not aligned. That is, the researchers expected students to engage in extensive reading and to have some interest in the articles; however, some learners spent too much time reading a single article and looking up every word, while others sped through the articles as fast as they could just to get done with the articles. This study is important because it shows that CALL design involves ‘getting into the heads of the learners’ and being able to anticipate how they will use the program. Certainly, this is no easy task.
Current research includes some ideas for designing CALL to accommodate many individual differences. I say ‘many’ here because I know that it is never possible to please every learner. Firstly, research highlights the importance of having a user-friendly and contemporary-looking interface (Nesselhauf & Tschichold, 2002). Although factors such as a programs’ looks may not appear to be all that important, learners believe otherwise, and will often choose a program based on how it looks as opposed to the program’s true merit. Secondly, Tozcu and Coady (2004) mention that learners in their study can individualize their own word lists while using the New Lexis software. This was not the main goal of the article, but the idea of allowing users to manipulate and individualize some aspects of their vocabulary-learning experience seems to be a good feature to add to any CALL program to accommodate individual differences. Lastly, a CALL vocabulary tutor (especially one that promotes vocabulary acquisition through extensive reading) should allow learners to select texts based on interest (Juffs et al., submitted; Renandya & Jacobs, 2002). Accommodating student interest is important for any extensive reading program because interesting readings can provide students with the necessary intrinsic motivation to read large amounts of texts.
Past research on the use of CALL vocabulary tutors has revealed the importance of considering individual differences when designing CALL applications. To some extent, it is impossible to predict exactly how students will use a CALL application, but it is important to pilot test new tutors and to interview students who use the program to understand students’ attitudes towards the program and predict problems that students might have. This research also indicates that a tutor’s looks, ease of use, and ability to be appropriated for a learner’s individual use are important factors to consider for designing CALL vocabulary tutors. Furthermore, CALL should allow learners to choose texts that are of interest to individual learners.
Conclusions: Best Practices for Designing CALL Vocabulary Tutors
In light of the previously cited research, some general conclusions can be drawn in regards to ‘best practices’ for the design of vocabulary tutors. Firstly, CALL vocabulary tutors should teach words that students need to learn in order to enter into academic programs and make them noticeable for language learners. Words that come from the AWL or another well-known word list should be taught, as well as frequent multi-word units such as phrasal verbs and collocations. Current research has shown that concordancers or bilingual concordancers may be useful in promoting knowledge and recognition of multi-word units. Secondly, CALL tutors should: 1) provide immediate feedback to learners, 2) allow optimally spaced rehearsal, and 3) provide learners with L1 translations so that learners can verify the meanings of words in their L1 before committing them to memory. It is argued that all of these factors lead to deep-processing, which is linked to long-term retention. Thirdly, CALL vocabulary tutors should include features which allow learners to interact with other learners while or immediately after using CALL. Lastly, CALL designers must consider the role of individual differences in vocabulary learning from CALL. In order to accommodate individual differences, CALL programs should be easy to use, modern-looking, and customizable, while at the same time providing learners with interesting materials to learn from.

References
Allum, P. (2004). Evaluation of CALL: Initial vocabulary learning. ReCALL, 16(2), 488-501.
Chan, T., & Liou, H. (2005). Effects of web-based concordancing instruction on EFL students' learning of verb-noun collocations. Computer Assisted Language Learning, 18(3), 231-250.
Cobb, T. (1999). Breadth and depth of lexical acquisition with hands-on concordancing. Computer Assisted Language Learning, 12(4), 345-360.
Coxhead, A. (2001) A new academic word list. TESOL Quarterly 34: 213-238.
De la Fuente, M. (2003). Is SLA interactionist theory relevant to CALL? A study on the effects of computer-mediated interaction in L2 vocabulary acquisition. Computer Assisted Language Learning, 16(1), 47-81.
Dörnyei, Z., & Skehan, P. (2005). Individual differences in second language learning. In C. Doughty & M. Long (Eds.), The handbook of second language acquisition. Massachusetts: Blackwell.
Folse, K. (2006). The effect of type of written exercise on L2 vocabulary retention. TESOL Quarterly, 40(2), 273-293.
Grabe, W. (2002). Dilemmas for the development of second language reading abilities. In J. C. Richards & W. A. Renandya (Eds.), Methodology in language teaching. New York: Cambridge University Press.
Grace, C. A. (1998). Retention of word meanings inferred from context and sentence-level translations: Implications for the design of beginning-level CALL software. The Modern Language Journal, 82(iv), 533-544.

Groot, P. (2000). Computer assisted second language vocabulary acquisition. Language Learning and Technology, 4, 60-81.
Horst, M. (2005). Learning L2 vocabulary through extensive reading: A measurement study. The Canadian Modern Language Review, 61(3), 355-382.
Hunt, A., & Beglar, D. (2002). Current research and practice in teaching vocabulary. In J. C. Richards & W. A. Renandya (Eds.), Methodology in language teaching. New York: Cambridge University Press.
Juffs, A., Friedline, B., Wilson, L., Eskenazi, M., & Heilman, M. (submitted). Activity theory and computer-assisted learning of English vocabulary. Applied Linguistics.
Mitchell, R., & Myles, F. (2004). Second language learning theories (2nd ed.). New York: Hodder Arnold.
Nakata, T. (2008). English vocabulary learning with word lists, word cards and computers: Implications from cognitive psychology research for optimal spaced learning. ReCALL, 20(1), 3-20.
Nation, P. (2002). Best practice in vocabulary teaching and learning. In J. C. Richards & W. A. Renandya (Eds.), Methodology in language teaching. New York: Cambridge University Press.
Nesselhauf, N., & Tschichold, C. (2002). Collocations in CALL: An investigation of vocabulary-building software for EFL. Computer Assisted Language Learning, 15(3), 251-279.
Renandya, W., & Jacobs, G. (2002). Extensive reading: Why aren't we all doing it? In J. C. Richards & W. A. Renandya (Eds.), Methodology in language teaching. New York: Cambridge University Press.

Smith, B. (2004). Computer-mediated negotiated interaction and lexical acquisition. SSLA, 26, 365-398.
Sun, Y., & Wang, L. (2003). Concordancers in the EFL classroom: Cognitive approaches and collocation difficulty. Computer Assisted Language Learning, 16(1), 83-94.
Tozcu, A., & Coady, J. (2004). Successful learning of frequent vocabulary through CALL also benefits reading comprehension speed. Computer Assisted Language Learning, 17(5), 473-495.
Zapata, G., & Sagarra, N. (2007). CALL on hold: The delayed benefits of an online workbook on L2 vocabulary learning. Computer Assisted Language Learning, 20(2), 153-171.

Cognitive Factors

2009-11-22T17:42:50Z

Gparent:

The research in this thrust is aimed at understanding cognitive learning—changes in knowledge—that result from [[instructional events]]. It builds on work in the learning sciences field at large and on research carried out in the PSLC over its first four years within the [[Refinement and Fluency]] cluster and part of the [[Coordinative Learning]] cluster, thereby merging two themes that organized the first phase of the PSLC. Each of these clusters was concerned with identifying instructional events that produce robust learning. They differed mainly in that the relevant theme within the Coordinative Learning cluster had a specific focus on instructional events that included more than one input. (A second theme within the Coordinative Learning cluster was on instructional events that provoke learning events involving more than one reasoning method and this theme will be continued in the [[Metacognition and Motivation]] thrust). In the fifth year of the PSLC, we carry forward research from each of these clusters, while making a transition to an additional set of research questions. Although we frame this section in terms of the new Cognitive Factors thrust, the research carried out during the 5th year has been initiated in the current Refinement and Fluency and in part of the Coordinative Learning clusters.

Our work on cognitive factors encompasses a triangulated set of events around learning: learning events, instructional events, and assessment events. Anything from a lesson to an entire curriculum can be considered a sequence of events whose durations vary from seconds to semesters. The hypotheses of the Cognitive Factors Thrust concern how instructional procedures (e.g., decisions about the learner’s task, materials, practice, feedback) affect learning events and thus the outcomes of learning. Learning involves the acquisition of [[knowledge components]], an increase in the [[feature validity]] and the [[strength]] of these components, and the integration of these components through practice. Our basic hypotheses include the following:

* Explicitness: Instruction that draws the learner’s attention to valid features that support the relevant knowledge components leads to more robust learning than instruction that does not.
* Assistance: The degree of assistance in the instruction affects learning in relation to student knowledge on specific knowledge components.
* Practice: Practice schedules can be optimized using models of learning based on memory activation assumptions.
* Integration: Knowledge components that are integrated during learning and practice lead to more robust learning and fluent performance across different tasks.

The research plan tests these hypotheses across knowledge domains, as exemplified by the following projects:

''Language background factors in L2 learning''. This work illustrates the synergies that develop in the PSLC’s LearnLab context, in this case between English as a second language (ESL) director Alan Juffs and other PSLC language researchers. In a prior cluster meeting, Juffs presented ESL classroom data that compared various L1 background students in their performance on transcribing their own speech, a standard piece of instruction in the ESL curriculum. The result that caught the interest of PSLC researchers (Dunlap, Guan, Perfetti) was the very poor spelling performance of Arabic-background students, relative to Spanish, Korean, and Chinese ESL students, despite comparable levels of spoken language performance. Furthermore, Juffs identified this discrepancy as a long-standing one in ESL instruction. Although one might hypothesize that a key factor is orthographic differences between L1 and L2, this seems unlikely here. Spanish to English is closer, but Chinese to English is farther in L1-L2 orthographic similarity. The first steps toward a new study have been taken with the help of a PSLC summer intern, who coded the errors made in spelling by all L1 background learners. The pattern of errors can be characterized as qualitatively similar, differing across languages quantitatively, suggesting a generalized English spelling problem. This analysis has led to the hypothesis that feature focusing—attention to full spelling patterns—is different across the L1 backgrounds, which we will test in a training experiment that focuses attention on spelling patterns.

''Second language vocabulary learning''. Another new project originating within the Refinement and Fluency cluster will study English vocabulary learning using REAP. Based on recent research by Balass on the trade-offs between explicit (dictionary-based) and implicit (inferences from text) instruction in learning new words by monolingual subjects (Bolger et al, 2008), the new work will apply this tradeoff idea to second language learners. The hypothesis is that allowing learners to view definitions is more effective after they have read a sentence containing the word to be learned. This hypothesis reflects ideas about assistance (giving a definition versus inferring it) and the assumption that learning word meanings from context depends on the overlapping memory traces established by specific encounters with the word (Bolger et al, 2008). REAP allows us to use authentic texts for studies with students of various L1 backgrounds learning English through reading texts in their areas of interest. In our experiments, we will vary the availability of definitions provided on-line as part of the text reading.

''Explicit instruction and practice schedules in algebra and second language learning''. Foreign language learning in classrooms has stimulated research on explicit vs implicit instruction, with conclusions favoring the value of explicit instruction (Norris and Ortega, 2000). A major conclusion from PSLC work is that instruction that draws attention to critical valid features—“feature focusing”—is important in acquiring knowledge components for complex tasks. This conclusion has evidence from studies of L2 learning of the English grammar by Levin, Friskoff, Pavlik, studies of radical learning by Dunlap et al and by Pavlik, and by studies by Zhang and MacWhinney and by Liu et al on learning spoken syllables through pin-yin (alphabetic spellings). Projects in French dictation (MacWhinney) and French grammar (Presson & MacWhinney), Chinese dictation (Zhang & MacWhinney), algebra (Pavlik) and arithmetical computation (Fiez) also reflect this theme. Much of this work has been combined with completely general hypotheses about practice, based on Pavlik and Anderson (2005)’s model that describes the trade-off between the benefit of spaced practice and the cost of longer retention intervals brought by spacing. The resulting optimized practice schedule has been tested in several PSLC studies of vocabulary learning in Chinese (Pavlik, MacWhinney, Koedinger; reported in Pavlik, 2006), cues to French gender (Presson, MacWhinney, & Pavlik). Important is the generality of the optimization model. It applies to all domain content and studies in both algebra and second language learning have nee carried out. The new work in second language and in algebra builds on the synergies that have emerged from collaborations between domain researchers (e.g. MacWhinney) and Pavlik around experiments and models for optimizing practice. For Chinese, MacWhinney, Zhang, and Pavlik have developed a tutor for Chinese dictation and vocabulary learning that is being used in 18 sites. Data from these sites will be used to test the results of practice schedules and the form of instructional events (e.g. cues to gender in French) with longer term measures of robust learning. Because each of the tutors logs results to DataShop, the student records are a rich source of data for further study, including researchers beyond the PSLC.

''Learning the logic of unconfounded experiments.'' We will extend our research on college level science topics (chemistry and physics) to middle school science, with a focus on the cross-domain topic of experimental design. The ability to design unconfounded experiments and make valid inferences from their outcomes is an essential skill in scientific reasoning. The key idea here is CVS: the Control of Variables Strategy. CVS is the fundamental idea underlying the design of unconfounded experiments from which valid, causal, inferences can be made. Its acquisition is an important step in the development of scientific reasoning skills , because it provides a strong constraint on search in the space of experiments (Klahr, 2000). The Tutor for Experimental Design (TED), developed by Klahr’s research team, builds on previous work studying the different paths of learning and transfer that result from teaching CVS using different instructional methods that span from direct instruction to discovery (Chen & Klahr, 1999) and show differences along the “physical-virtual” dimension (Triona & Klahr, 2007). We build on this by constructing of a semi-autonomous tutor, then developing a full computer based tutor in Pittsburgh middle school LearnLabs and carrying out in vivo experiments with TED.

''Integration of knowledge components.'' Isolated knowledge components are not sufficient to produce fluent use of knowledge. Integrating knowledge components is important both in authentic practice that follows acquisition of knowledge components but, we hypothesize, also in the initial acquisition of components. Some of our prior work in coordinative learning establishes some of the conditions that favor multiple inputs during learning (e.g., Davenport et al in stochiometry). And experiments on fluency support the value of repeated practice in single-topic speaking as way to support fluency (de Jong, Halderman and Perfetti). In new work we propose to build on progress we have made in the study of fluency in language (de Jong et al) and arithmetic (Fiez). For example, we will follow the discovery by de Jong and colleagues that when L2 speakers repeat a speech on a single topic, their fluency scores increase on a number of measures. We will test the hypothesis that this results from the advantage of retrieving the same conceptual and lexical knowledge and overall speech plan on successive attempts, allowing fluency to increase on procedural components supported by chunking of words to phrases. We are accumulating a large database in the English LearnLab that will support the testing of additional hypotheses. The idea that some relatively simple learning (e.g. 3-5 knowledge components) is supported by integration from the beginning is being tested by Liu, Guan & Perfetti in a study of learning to read Chinese characters. The hypothesis is that when students write unfamiliar characters within the same 60-second time period that they read the character and try to learn its meaning and pronunciation, they will show more robust learning measured by reading tasks. Underlying this hypothesis is the idea that the representation of a character (or other objects that follow structural principles) can be perceptual-motor as well as visual.

== Descendents ==

*[[Klahr - TED]]
*[[Perfetti - Read Write Integration]]
*[[MacWhinney - Second Language Grammar]]
*[[Juffs - Feature Focus in Word Learning]]
*[[de Jong - Fluency]]
*[[McLaren_-_The_Assistance_Dilemma_And_Discovery_Learning | McLaren - The Assistance Dilemma and Discovery Learning]]
*[[Wylie - Intelligent Writing Tutor]]
*[[REAP_main | Eskenazi - REAP]]

=== References ===
* Borek, A., McLaren, B.M., Karabinos, M., & Yaron, D. (2009). How Much Assistance is Helpful to Students in Discovery Learning? In U. Cress, V. Dimitrova, & M. Specht (Eds.), Proceedings of the Fourth European Conference on Technology Enhanced Learning, Learning in the Synergy of Multiple Disciplines (EC-TEL 2009), LNCS 5794, September/October 2009, Nice, France. (pp. 391-404). Springer-Verlag Berlin Heidelberg.

REAP Multimodal Learning Fall 09

2009-11-09T18:19:28Z

Gparent:

REAP Multimodal Learning Fall 09

2009-11-09T18:16:06Z

Gparent: /* Hypotheses */

REAP Multimodal Learning Fall 09

2009-11-09T18:10:43Z

Gparent:

File:MultimodalLearning.JPG

2009-11-09T18:10:21Z

Gparent: Example of a REAP instruction with both auditory and visual KC

Example of a REAP instruction with both auditory and visual KC

REAP multimodal learning fall09

2009-11-09T18:03:09Z

Gparent: REAP multimodal learning fall09 moved to REAP Multimodal Learning Fall 09: Capitalized

#REDIRECT [[REAP Multimodal Learning Fall 09]]

REAP Multimodal Learning Fall 09

2009-11-09T18:03:09Z

Gparent: REAP multimodal learning fall09 moved to REAP Multimodal Learning Fall 09: Capitalized

REAP Multimodal Learning Fall 09

2009-11-09T18:02:18Z

Gparent:

REAP Multimodal Learning Fall 09

2009-11-09T17:52:39Z

Gparent:

REAP Multimodal Learning Fall 09

2009-11-09T17:45:12Z

Gparent:

== REAP Study on Multimodal Learning of Vocabulary==

=== Logistical Information ===

{| border="1"
|+
|-
| '''Contributors''' || Gabriel Parent, Luis Marujo, Adam Skory, Maxine Eskenazi
|-
| '''Study Start Date''' || October 13, 2009
|-
| '''Study End Date''' || November 20, 2009
|-
| '''Learnlab Courses''' || English Language Institute Reading 4 and 5 (ESL LearnLab)
|-
| '''Number of Students''' || 68
|-
| '''Total Participant Readings (est.)''' || 816
|-
| '''Data in Datashop''' || no
|}

=== Abstract ===

The term “Multimodal learning” refers to learning where two or more different modes are used in order to integrate a [[Knowledge Component]]. In the particular case of vocabulary, the typical modes would be:

Word learning episodes can occur through writing and speech. Spoken language knowledge is important as a foundation for reading, and may support word learning through reading. Under what conditions does spoken language input, when combined with written input, lead to better word learning for L2 learners?

[[Image:Reap context definition.jpg|500px]]

{| border="1"
|+
|-
| ||'''Passive''' || '''Active''' || '''Interactive'''
|-
| '''Explicit (general)''' || Dictionary Definitions || || Practice Exercises
|-
| '''Implicit (instance)''' || Interpreting meaning in context while reading || Sentence Production (assessment) || Practice Exercises
|}

=== Glossary ===

''Intrinsic Motivation:'' Motivation to learn for learning's own sake rather than some external goal.

''Extrinsic Motivation:'' Motivation for learn in order to satisfy an external goal, such as completing a task or passing an assessment.

=== Research question ===

Does [[personalization]] of practice readings to match students' personal interests increase ESL vocabulary learning?

=== Dependent variables ===
[[Normal post-test]] scores

[[Normal post-test]] scores for practiced words only

[[Long-term retention]] test scores, same post-test but administered months later.

Evidence of [[Transfer]]: sentence production tasks for target words, correct use of words in writing assignments for other courses.

=== Independent variables ===
[[Personalization]] of readings by topics of interest. In the control condition, the tutor did not use potential personal interest as a factor in its selection of reading materials. In the treatment condition, the tutor did use interest as a factor. All other selection criteria were the same in both conditions. Time on task was also the same.

=== Hypotheses ===

Since intrinsic motivation seems to be important in language learning, the benefits of [[personalization]] will outweigh the costs.

=== Findings ===

Personalization to match interests can lead to improved learning of the relevant knowledge components in a tutoring environment for vocabulary learning. Students in the treatment group correctly answered a higher proportion of questions on target words that were practiced in the REAP tutor.

Second, personalization can compromise domain-based goals. In the REAP tutor, an important domain-based goal is to give the student practice opportunities for many new target words. However, students receiving personalization practiced fewer target words. The difficulty in achieving the domain-based goal of practicing many unknown words is due to the fact that the REAP tutor often could not find texts that included multiple target words and also matched personal interests.

Third and finally, if the challenges of negotiating personalization and domain-based goals are met, personalization can lead to improvements in overall learning. Students with personalization appeared to learn the words they practiced with greater frequency but practiced fewer target words, and as a result did not perform reliably differently than their controls on the overall post-test measure for cloze questions. The researchers attributed this lack of a difference to the fact that, in many cases, the tutor had to choose between interesting readings and those with more practice opportunities. However, the availability of readings that are both interesting and provide ample practice is a technical issue which can be solved in a straightforward manner by increasing the size and coverage of the corpus of available practice reading passages.

=== Explanation ===

Students in the treatment condition with [[personalization]] performed slightly better on average (M=35.5%, SD=14.9%) in terms of overall post-test scores compared to students in the control condition (M=27.1%, SD=17.2%). However, this difference was not statistically significant.

[[Image:graph40.jpg|500px]]

There is evidence that the difference in post-test scores is due to increased interest leading to deeper processing of the reading practice texts.

Responses to questionnaires following each reading show the interest level of students using the REAP tutor. The questionnaires asked students to indicate on a scale from one to five their interest inr the preceding text. The distributions of post-reading interest ratings for students in the treatment and control conditions are shown below.

[[Image:Interest_combined.jpg|700px]]

Students were also given an exit survey during their last week of practice with the tutor that asked them, among other questions, for to indicate whether they agreed with the statement, “Most of the readings were interesting.” The ratings were on a scale from one to five, with five indicating strong agreement and one indicating strong disagreement. Exit survey interest ratings by students in the treatment condition were significantly higher (p<0.05) than the ratings by students in the control condition. The mean response for students who received personalized readings was 3.18, while it was 2.65 for students in the control condition.

Further analysis of post-test scores reveals that students did learn more of the words that they actually practiced in REAP. The post-test contained 40 questions for target vocabulary words. Many of the students did not practice 40 words, so performance on practiced words alone was analyzed. Students in the treatment condition scored higher (N=16, M=50.3, SD=20.1) on questions for words seen in readings than did students in the control condition (N=19, M=32.4, SD=18.9). A two-tailed t-test for independent means verified that this result is statistically significant (t=2.719, df=33, p=0.005). The difference of scores between the two groups was 17.9% (95% CI = 4.5%, 31.3%), which corresponds to a large effect size of 0.85. This result indicates that [[personalization]] improved learning for the words that students saw in readings, which is in line with previous findings that intrinsic motivation leads to improved learning.

[[Image:post_just_practice.jpg|400px]]

However, students in the treatment condition that included [[personalization]] saw fewer words in their training sessions (N=16, M=12.0 , SD=1.13) than students in the control condition (N=19, M=16.3, SD=0.87) (t=-2.9, df=33, p=0.006). Average time on task was essentially the same for students in both conditions. Students in the treatment condition spent slightly longer on each reading. The main reason, however, for the difference in the average total number of words practiced was that students for whom the tutor provided personalized instruction saw fewer words (M=3.41, SD=0.55) per practice reading passage than students in the control condition (M=4.07, SD=0.83) (t=2.929, df=33, p=0.006).

Thus, when the tutor used [[personalization]] as a factor in the selection of readings, it chose readings that were less valuable according to other factors. Specifically, this result shows that by personalizing instruction, the tutor was not able to provide practice for as many words. Of course, the practice that it did provide was better, as is shown in the previous result that for words student did practice, [[personalization]] appeared to increase learning.

The reduced number of target words per text with personalization is a technical issue which can be avoided in a straightforward manner by increasing the size of the database of readings. With more readings, the tutor can find texts that both have ample target words and cover topics of personal interest.

[[Image:words_per_reading.jpg|400px]]

There is a possibility that the students in the treatment condition who were seeing fewer words in each reading were learning more of the words simply because they had fewer to learn per reading. To rule out this hypothesis, regression analyses (multiple linear regression) with overall post-test performance and performance for practiced words as the dependent variables. In both regression analyses, the number of target words per reading was not a significant predictor of performance. In fact, the number of target words per document was slightly positively correlated with post-test performance in both cases. This result seems to rule out the possibility that students were learning more target words in the treatment condition because they were seeing fewer words.

[[Long-term retention]] test results showed no reliable differences because of a small sample size. The test was administered to students who stayed in the ELI in the subsequent semester, which constituted only a fraction of the original sample.

=== Further Information ===

The following study addresses a different form of personalization, by which interactions with the learner (e.g., instructions, directions) are conducted using casual and direct rather than formal language:

[[Stoichiometry_Study | Studying the Learning Effect of Personalization and Worked Examples in the Solving of Stoichiometry Problems (McLaren, Koedinger & Yaron)]]

=== Annotated bibliography ===

Note: a paper on this study has been submitted to International Journal of Artificial Intelligence in Education.

[http://reap.cs.cmu.edu/Papers/heilman_topic_choice_AIED2007_poster_final.pdf Heilman, M., Juffs, A., & Eskenazi, M. (2007). Choosing Reading Passages for Vocabulary Learning by Topic to Increase Intrinsic Motivation. Proceedings of the 13th International Conferenced on Artificial Intelligence in Education. Marina del Rey, CA. (poster)]

Clark, R. C. and Mayer, R. E. (2003). e-Learning and the Science of Instruction. Jossey-Bass/Pfeiffer.

Cordova, D. I. & Lepper, M. R. (1996). Intrinsic Motivation and the Process of Learning: Beneficial Effects of Contextualization, Personalization, and Choice. Journal of Educational Psychology. Vol. 88,l No. 4, 715-730.

Lepper, M. (1988). Motivational Considerations in the Study of Instruction. Cognition and Instruction. 5(4), 289-309.

Heilman, M., Juffs, A., & Eskenazi, M. (To Appear). Choosing Reading Passages for Vocabulary Learning by Topic to Increase Intrinsic Motivation. Proceedings of the 13th International Conferenced on Artificial Intelligence in Education. Marina del Rey, CA. (poster)

REAP Multimodal Learning Fall 09

2009-11-09T17:36:03Z

Gparent: New page: == REAP Study on Multimodal Learning of Vocabulary== === Logistical Information === {| border="1" |+ |- | '''Contributors''' || Gabriel Parent, Luis Marujo, Adam Skory, Maxine Eskenazi...

== REAP Study on Multimodal Learning of Vocabulary==

=== Logistical Information ===

{| border="1"
|+
|-
| '''Contributors''' || Gabriel Parent, Luis Marujo, Adam Skory, Maxine Eskenazi
|-
| '''Study Start Date''' || September 11, 2006
|-
| '''Study End Date''' || November 21, 2006
|-
| '''Learnlab Courses''' || English Language Institute Reading 4 (ESL LearnLab)
|-
| '''Number of Students''' || 35
|-
| '''Total Participant Hours (est.)''' || 270
|-
| '''Data in Datashop''' || no
|}

=== Abstract ===

In this work, the term “[[personalization]]” refers to the selection of practice readings in order to match a student’s interests.

During each training session with the REAP Tutor, students work through a series of readings, each of which is followed by
practice exercises for the target words in the reading. While reading a passage, students are able to access
dictionary definitions for any word in a reading either by clicking on a highlighted target word or by typing a
word into a box in the lower-left corner of the screen. The target words in the readings are also highlighted
because highlighting may increase the use of dictionary definitions, thus encouraging students to
coordinate multiple sources of information about a word’s meaning—namely, the implicit examples from context around
words and the explicit generalizations in the definitions of words (as exemplified in the figure below)

[[Image:Reap context definition.jpg|500px]]

A problem discovered in past studies with REAP is that many students spend only a brief amount of time
on a reading and do not deeply process the text. Students often only read the dictionary definition for target
words rather than attempting to process the entire context around the words. Inferring the meaning of
vocabulary from context is a seemingly important strategy that is not used by such students. This behavior is likely due to a desire to perform well on post-reading practice exercises and post-test, which can be viewed as forms of extrinsic motivation. Intrinsically
motivated students who are more interested in a reading are more likely to read the entire text and to use
context to learn the meaning of unknown vocabulary. Therefore, [[personalization]] that increases intrinsic
motivation could lead to deeper processing of context and better learning of vocabulary.

{| border="1"
|+
|-
| ||'''Passive''' || '''Active''' || '''Interactive'''
|-
| '''Explicit (general)''' || Dictionary Definitions || || Practice Exercises
|-
| '''Implicit (instance)''' || Interpreting meaning in context while reading || Sentence Production (assessment) || Practice Exercises
|}

=== Glossary ===

''Intrinsic Motivation:'' Motivation to learn for learning's own sake rather than some external goal.

''Extrinsic Motivation:'' Motivation for learn in order to satisfy an external goal, such as completing a task or passing an assessment.

=== Research question ===

Does [[personalization]] of practice readings to match students' personal interests increase ESL vocabulary learning?

=== Dependent variables ===
[[Normal post-test]] scores

[[Normal post-test]] scores for practiced words only

[[Long-term retention]] test scores, same post-test but administered months later.

Evidence of [[Transfer]]: sentence production tasks for target words, correct use of words in writing assignments for other courses.

=== Independent variables ===
[[Personalization]] of readings by topics of interest. In the control condition, the tutor did not use potential personal interest as a factor in its selection of reading materials. In the treatment condition, the tutor did use interest as a factor. All other selection criteria were the same in both conditions. Time on task was also the same.

=== Hypotheses ===

Since intrinsic motivation seems to be important in language learning, the benefits of [[personalization]] will outweigh the costs.

=== Findings ===

Personalization to match interests can lead to improved learning of the relevant knowledge components in a tutoring environment for vocabulary learning. Students in the treatment group correctly answered a higher proportion of questions on target words that were practiced in the REAP tutor.

Second, personalization can compromise domain-based goals. In the REAP tutor, an important domain-based goal is to give the student practice opportunities for many new target words. However, students receiving personalization practiced fewer target words. The difficulty in achieving the domain-based goal of practicing many unknown words is due to the fact that the REAP tutor often could not find texts that included multiple target words and also matched personal interests.

Third and finally, if the challenges of negotiating personalization and domain-based goals are met, personalization can lead to improvements in overall learning. Students with personalization appeared to learn the words they practiced with greater frequency but practiced fewer target words, and as a result did not perform reliably differently than their controls on the overall post-test measure for cloze questions. The researchers attributed this lack of a difference to the fact that, in many cases, the tutor had to choose between interesting readings and those with more practice opportunities. However, the availability of readings that are both interesting and provide ample practice is a technical issue which can be solved in a straightforward manner by increasing the size and coverage of the corpus of available practice reading passages.

=== Explanation ===

Students in the treatment condition with [[personalization]] performed slightly better on average (M=35.5%, SD=14.9%) in terms of overall post-test scores compared to students in the control condition (M=27.1%, SD=17.2%). However, this difference was not statistically significant.

[[Image:graph40.jpg|500px]]

There is evidence that the difference in post-test scores is due to increased interest leading to deeper processing of the reading practice texts.

Responses to questionnaires following each reading show the interest level of students using the REAP tutor. The questionnaires asked students to indicate on a scale from one to five their interest inr the preceding text. The distributions of post-reading interest ratings for students in the treatment and control conditions are shown below.

[[Image:Interest_combined.jpg|700px]]

Students were also given an exit survey during their last week of practice with the tutor that asked them, among other questions, for to indicate whether they agreed with the statement, “Most of the readings were interesting.” The ratings were on a scale from one to five, with five indicating strong agreement and one indicating strong disagreement. Exit survey interest ratings by students in the treatment condition were significantly higher (p<0.05) than the ratings by students in the control condition. The mean response for students who received personalized readings was 3.18, while it was 2.65 for students in the control condition.

Further analysis of post-test scores reveals that students did learn more of the words that they actually practiced in REAP. The post-test contained 40 questions for target vocabulary words. Many of the students did not practice 40 words, so performance on practiced words alone was analyzed. Students in the treatment condition scored higher (N=16, M=50.3, SD=20.1) on questions for words seen in readings than did students in the control condition (N=19, M=32.4, SD=18.9). A two-tailed t-test for independent means verified that this result is statistically significant (t=2.719, df=33, p=0.005). The difference of scores between the two groups was 17.9% (95% CI = 4.5%, 31.3%), which corresponds to a large effect size of 0.85. This result indicates that [[personalization]] improved learning for the words that students saw in readings, which is in line with previous findings that intrinsic motivation leads to improved learning.

[[Image:post_just_practice.jpg|400px]]

However, students in the treatment condition that included [[personalization]] saw fewer words in their training sessions (N=16, M=12.0 , SD=1.13) than students in the control condition (N=19, M=16.3, SD=0.87) (t=-2.9, df=33, p=0.006). Average time on task was essentially the same for students in both conditions. Students in the treatment condition spent slightly longer on each reading. The main reason, however, for the difference in the average total number of words practiced was that students for whom the tutor provided personalized instruction saw fewer words (M=3.41, SD=0.55) per practice reading passage than students in the control condition (M=4.07, SD=0.83) (t=2.929, df=33, p=0.006).

Thus, when the tutor used [[personalization]] as a factor in the selection of readings, it chose readings that were less valuable according to other factors. Specifically, this result shows that by personalizing instruction, the tutor was not able to provide practice for as many words. Of course, the practice that it did provide was better, as is shown in the previous result that for words student did practice, [[personalization]] appeared to increase learning.

The reduced number of target words per text with personalization is a technical issue which can be avoided in a straightforward manner by increasing the size of the database of readings. With more readings, the tutor can find texts that both have ample target words and cover topics of personal interest.

[[Image:words_per_reading.jpg|400px]]

There is a possibility that the students in the treatment condition who were seeing fewer words in each reading were learning more of the words simply because they had fewer to learn per reading. To rule out this hypothesis, regression analyses (multiple linear regression) with overall post-test performance and performance for practiced words as the dependent variables. In both regression analyses, the number of target words per reading was not a significant predictor of performance. In fact, the number of target words per document was slightly positively correlated with post-test performance in both cases. This result seems to rule out the possibility that students were learning more target words in the treatment condition because they were seeing fewer words.

[[Long-term retention]] test results showed no reliable differences because of a small sample size. The test was administered to students who stayed in the ELI in the subsequent semester, which constituted only a fraction of the original sample.

=== Further Information ===

The following study addresses a different form of personalization, by which interactions with the learner (e.g., instructions, directions) are conducted using casual and direct rather than formal language:

[[Stoichiometry_Study | Studying the Learning Effect of Personalization and Worked Examples in the Solving of Stoichiometry Problems (McLaren, Koedinger & Yaron)]]

=== Annotated bibliography ===

Note: a paper on this study has been submitted to International Journal of Artificial Intelligence in Education.

[http://reap.cs.cmu.edu/Papers/heilman_topic_choice_AIED2007_poster_final.pdf Heilman, M., Juffs, A., & Eskenazi, M. (2007). Choosing Reading Passages for Vocabulary Learning by Topic to Increase Intrinsic Motivation. Proceedings of the 13th International Conferenced on Artificial Intelligence in Education. Marina del Rey, CA. (poster)]

Clark, R. C. and Mayer, R. E. (2003). e-Learning and the Science of Instruction. Jossey-Bass/Pfeiffer.

Cordova, D. I. & Lepper, M. R. (1996). Intrinsic Motivation and the Process of Learning: Beneficial Effects of Contextualization, Personalization, and Choice. Journal of Educational Psychology. Vol. 88,l No. 4, 715-730.

Lepper, M. (1988). Motivational Considerations in the Study of Instruction. Cognition and Instruction. 5(4), 289-309.

Heilman, M., Juffs, A., & Eskenazi, M. (To Appear). Choosing Reading Passages for Vocabulary Learning by Topic to Increase Intrinsic Motivation. Proceedings of the 13th International Conferenced on Artificial Intelligence in Education. Marina del Rey, CA. (poster)