Assistance Dilemma English Articles
| PI | Ruth Wylie, Teruko Mitamura, Ken Koedinger |
|---|---|
| Others with > 160 hours | Jim Rankin |
| Start date study 1 | September 2008 |
| End date study 1 | December 2008 |
| Start date study 2 | January 2009 |
| End date study 2 | May 2009 |
| Learnlab | English |
Contents
Intelligent Writing Tutor
Ruth Wylie, Teruko Mitamura, Ken Koedinger, and Jim Rankin
Background
This project focuses on integrated theory development with respect to the Assistance Dilemma (Koedinger and Aleven, 2007), specifically the instructional principle of self-explanation. We explore whether having students self-explain and thereby adding difficulty aids or harms eventual learning outcomes.
This work sets out to test, for the first time, whether the success of self-explanation extend to second language grammar learning, particularly the English article system (teaching students the difference between when to say “a dog” vs “the dog”). Previous work has demonstrated the effectiveness of self-explanation to enhance robust learning across multiple math and science domains and a variety of learners. Our goal is to better understand how, why, and when self-explanation works by investigating the effects of self-explanation on second language acquisition.
Self-Explanation Study 1
Hypothesis
Since the added process of self-explanation is generative, students in the self-explanation with menu condition will show greater learning gains on both normal and robust learning measures than those in the practice-only condition and free-form self-explanation condition. We hypothesize that the added difficulty of generating explanations in the free-form condition is extraneous and thus will result in less learning than the menu-based self-explanation conditions.
Alternatively, generating self-explanations may require too much time and thus it may be better to provide students with extra examples and to implicitly learn the rules through the practice-only condition.
In addition, while self-explanation has proved to lead to greater learning in other domains, this would be the one of the first, if not the first, study to empirically examine its effects in second language grammar acquisition.
Independent Variables
Image 1: Practice-Only Tutor - Using this tutor, students select an article (a, an, the, no article) from each sentence to complete the sentence.
Image 2: Menu-based Self-Explanation Tutor - Using this tutor, students select an article (a, an, the, no article) from each sentence and choose the feature of the sentence that best explains the article choice.
Image 3: Free-form Self-Explanation Tutor- Using this tutor, students select an article (a, an, the, no article) from each sentence and write an explanation for why they made their choice.
Dependent Variables
All students completed a computer-based pre and posttest that consisted of article-only and article with explanation items. In the article-only items, students chose an article from a dropdown menu to complete the sentence. In the article with explanation items, students chose an article to complete the sentence and then chose the feature or rule that explained their answer. No hints were available during the tests, and students did not receive feedback on their answers.
Results
Students (n=63) in all conditions showed significant pre to posttest improvement. While there were no significant main effects of condition, there was a significant aptitude-treatment interaction (F(2, 60) = 3.54, p = 0.036) of theoretical interest. The dependent variable in this analysis was a learning efficiency score, which is a measure that combines learning gains and time-on-task into a single construct, and the article pre-test was the aptitude measure. As shown in the figure below, lower aptitude students benefited more (i.e., learned more in less time) from the self-explanation manipulations whereas higher aptitude students benefited more from the practice-only condition. This interaction is consistent with the theoretical notion that self-explanation is beneficial primarily during early construction and refinement of knowledge components whereas practice alone is better during knowledge strengthening and fluency development. It is also consistent with research on the worked example principle and particularly the "expertise reversal effect" (Kalyuga, Chandler, Tuovinen, & Sweller, 2001).
Self Explanation Study 2
The second study, conducted in Spring 2009 in Levels 3, 4, and 5 of the English LearnLab compared a practice-only tutoring system to a worked examples with self-explanation system. The goal of this study was to further investigate the effects of self-explanation on second language grammar learning. In the self-explanation conditions in the previous study, students were required to select an article and select an explanation. In this study, we isolated the two tasks such that students were either selecting articles (practice-only) or selecting an exlanation (worked examples with self-explanation).
Independent Variables
Practice-Only Tutor: Working with the practice-only tutor, students select an article (a, an, the, or no article) from a drop-down. Students receive immediate feedback on their selections and can request hints to help choose the right answer.
Worked Example with Self-Explanation Tutor:In the worked example plus self-explanation tutor, students are given a completed sentence with the target article highlighted and are asked to select the feature of the sentence that best explains the article choice.
Dependent Variables
Similar to Study 1, students were given a pretest and immediate posttest with items similar to those on which they were tutored. The pre and posttests contained both article selection and explanation selection items.
Results
Results showed that students (n=101) in both conditions showed significant learning gains (F(1, 99) = 40.28, p < 0.001) but there was no difference between conditions.
Contributions
These studies are the first to experimentally test the effects of self-explanation in second language grammar learning, and the current results suggest that there are limitations to the benefits of self-explanation. This work highlights the need to continue investigating the self-explanation effect in new and different domains. It suggests there may be limitations to its applicability. Additionally, it is important to understand the source of difficulty within a domain and identify how self-explanations may or may not address it. More research is needed to further specify those boundary conditions and relate them to basic understanding of cognitive processes.
References
1. Bjork, R. A. (1994). Memory and metamemory considerations in the training of human beings. In J. Metcalfe and A. Shimamura (Eds.), Metacognition: Knowing about knowing (pp.185-205). Cambridge, MA: MIT Press.
2. Celce-Murcia, Marianne, and Larsen-Freeman, Diane. (1983). The Grammar Book: An ESL/EFL Teacher’s Course. Rowley, Massachusetts: Newbury House Publishers.
3. Elson, Allegra B. Fossilized language forms: Implications and a search for solutions in an adult English as second language classroom. A PALPIN Inquiry Project, 2004.
4. Ericsson, K. A, & Simon, H. (1984). Protocol analysis: Verbal reports as data. Cambridge, MA: MIT Press
5. Hawkins, J. A. (1991). On (in) definite articles: implicatures and (un) grammaticality prediction. Journal of Linguistics(27), 405-442.
6. Knowledge Component Hypothesis (2007, April 30). In PSLC Theory Wiki. Retrieved March 27, 2008, from http://learnlab.org/research/wiki/index.php/Knowledge_component_hypothesis
7. Koedinger, K., & Aleven, V. (2007). Exploring the assistance dilemma in experiments with Cognitive Tutors. Educational Psychology Review. 19(3) 239-264.
8. Level 4 Writing Objectives. (2007, Novermber 13). Retrieved March 19, 2008, from http://learnlab.org/learnlabs/english/Level_4_Objectives/Objectives_Wr4fin.dco
9. Liu, D. & Gleason, J. (2002). Acquisition of the Article THE by Nonnative Speakers of English, An Analysis of Four Nongeneric Uses. SSLA, 24, 1-26.
10. Master, P. (1997). The English Article System: Acquisition, Function, and Pedagogy. System. 25,(2) 215-232.
11. Pavlik Jr., P.I., and Anderson, J. R. (2005). Practice and forgetting effects on vocabulary memory: An activation-based model of the spacing effect. Cognitive Science, 29(4), 559-586.
12. Schooler, JW & Engstler-Schooler, TY. (1990). Verbal overshadowing of visual memories: some things are better left unsaid. Cognitive Psychology. 22(1):36-71.
13. VanLehn, K., Siler, S., Murray, C., Yamauchi, T., & Baggett, W.B. (2003). Why do only some events cause learning during human tutoring? Cognition and Instruction, 21(3), 209-249.
14. Wylie, R. (2007) Are we asking the right questions? Understanding which tasks lead to the robust learning of English grammar. Accepted as a Young Researchers Track paper at the 13th International Conference on Artificial Intelligence in Education. Marina del Rey, California. July 9 – 13, 2007.
15. Wylie, R. (2008) Putting a/the stake in the ground: Making a priori predictions of student learning. Accepted as a Young Researchers Track paper at Intelligent Tutoring Systems 2008. Montreal, Canada. June 23 – 27, 2008.
