Theory Wiki - User contributions [en]

Wylie - Intelligent Writing Tutor

2011-02-14T15:26:50Z

Ruth-Wylie: /* Publications */

== '''Project Title: Generalizing Self-Explanation''' ==
Report prepared for PSLC Site Visit, January 2011 
PSLC Research Thrust(s): Cognitive Factors 
PSLC Learnlab Course(s): English as a Second Language

{| class="wikitable" border=1
|-
| width="150pt" | PI: Ruth Wylie Department: HCII Organization: CMU
| width="150pt" | PI: Ken Koedinger Department: HCII Organization: CMU
| width="150pt" | PI: Teruko Mitamura Department: HCII Organization: CMU
|}

Others who have contributed 160 hours or more: 
PSLC Undergraduate Intern: Melissa Sheng, Rice University

{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''Study Name/Number''' || Pilot Study: Modalities of Self-Explanation || Study 1: Practice vs. Self-Explanation Only || Study 2: Analogy, Explanation, and Practice || Study 3: Worked Examples, Explanation, and Practice
|-
| '''Study Date''' || Fall 2008 || Spring 2009 || Fall 2009 || Spring 2010
|-
| '''LearnLab Site''' || University of Pittsburgh, ELI || University of Pittsburgh, ELI || University of Pittsburgh, ELI || University of Pittsburgh, ELI
|-
| '''LearnLab Course''' || ESL || ESL || ESL || ESL
|-
| '''Number of Students''' || 63 || 118 || 99 || 97
|-
|-
| '''DataShop''' ||
* '''Pre/Post Test Score Data:''' Yes
* '''Paper or Online Tests:''' Online
* '''Scanned Paper Tests:''' n/a
* '''Blank Tests:''' uploaded soon
* '''Answer Keys: ''' uploaded soon
||
* '''Pre/Post Test Score Data:''' Yes
* '''Paper or Online Tests:''' Online
* '''Scanned Paper Tests:''' n/a
* '''Blank Tests:''' uploaded soon
* '''Answer Keys: ''' uploaded soon
||
* '''Pre/Post Test Score Data:''' Yes
* '''Paper or Online Tests:''' Online
* '''Scanned Paper Tests:''' n/a
* '''Blank Tests:''' uploaded soon
* '''Answer Keys: ''' uploaded soon
||
* '''Pre/Post Test Score Data:''' Yes
* '''Paper or Online Tests:''' Online
* '''Scanned Paper Tests:''' n/a
* '''Blank Tests:''' uploaded soon
* '''Answer Keys: ''' uploaded soon
|}

{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''Study Name/Number''' || Study 4: Adaptive Self-Explanation Tutor
|-
| '''Study Date''' || Fall 2010
|-
| '''LearnLab Site''' || University of Pittsburgh, ELI
|-
| '''LearnLab Course''' || ESL
|-
| '''Number of Students''' || 131
|-
|-
| '''DataShop''' ||
* '''Pre/Post Test Score Data:''' Yes
* '''Paper or Online Tests:''' Online
* '''Scanned Paper Tests:''' n/a
* '''Blank Tests:''' uploaded soon
* '''Answer Keys: ''' uploaded soon
|}

== Project Plan Abstract ==
Prompting students to self-explain during problem solving has proven to be an effective instructional strategy across many domains. However, despite being called a “domain general” strategy, very little work has been done in areas outside of math and science. Thus, it remains an open question whether the self-explanation effect will hold in new and different domains like second language grammar learning. Through a series of classroom studies, we compare the effects of four tutoring systems: two that facilitate deep processing (prompted self-explanation and analogical comparison) and two that increase the rate of processing (example study and practice). Results show that while deep processing in general, and self-explanation specifically, is an effective strategy, it may not be an efficient approach compared to practice alone. These results suggest that the benefits of self-explanation over traditional practice may not be truly domain independent.

== Achievements ==
===Findings===
We have conducted four in-vivo studies with plans to do a fifth study during Fall 2010. The first study compared two different forms of self-explanation to a ecologically valid control condition of practice-only. We looked at both menu-based self-explanations (students chose the rule from a provided menu) and free-form responses (students typed their explanation into an empty text box). The second study compared self-explanation only to practice only in order to understand the effects that self-explanation alone had on student learning. Studies 3 and 4 introduced new instructional manipulations with goals to both increase learning and efficiency. Study 3 used analogical comparisons in an attempt to reduce the metalinguistic demands placed on the students, and Study 4 used example study in an attempt to reduce the amount of time students spent working with the tutor. The main finding from this series of studies is that while prompting students to self-explain does lead to significant learning gains, students who self-explain perform no better than students in a traditional practice condition. Furthermore, self-explanation requires significantly more time meaning it is inefficient compared to practice. These results differ from those found in previous self-explanation studies conducted largely in math and science domains.

{| class="wikitable"
|-
| [[Image:IWT_study1_learninggains.jpg]]
| [[Image:IWT_study2_learninggains.jpg]]
|-
| Fig. 1. Study 1 Learning Gains
| Fig. 2. Study 2 Learning Gains
|-
| [[Image:IWT_study3_learninggains.jpg]]
| [[Image:IWT_study4_learninggains.jpg]]
|-
| Fig. 3. Study 3 Learning Gains
| Fig. 4. Study 4 Learning Gains
|}
Figures 1-4: Across all four studies, students in all conditions show significant, but equal, learning gains. Students improve when prompted to self-explanation either alone (Study 2) or paired with tutored practice (Studies 1, 3 and 4), but there is no difference in learning between students who are prompted to self-explain and those that are not.

In addition to learning gain analysis, we have also compared efficiency scores across conditions for Studies 2, 3, and 4. Efficiency scores combine learning gains and time to complete instruction into a single measure. These results reveal a similar pattern: practice is much more efficient than self-explanation for learning the English article system.

{| class="wikitable"
|-
| [[Image:IWT_study2_efficiency.jpg]]
| [[Image:IWT_study3_efficiency.jpg]]
|-
| Fig. 5. Study 2 Efficiency Analysis
| Fig. 6. Study 3 Efficiency Analysis
|-
| [[Image:IWT_study4_efficiency.jpg]]
|-
| Fig. 7. Study 4 Learning Gains
|}
Figures 5-7: Efficiency Score Analysis. Across all studies, students in the practice-only condition were much more efficient than those in the self-explanation condition.

These studies address a key PSLC goal of identifying under what conditions instructional strategies lead to robust learning. Our results to date suggest that self-explanation may not be the best form of instruction for all domains. In addition to planning our final study, our current work investigates the effects of self-explanation prompts on robust learning measures like transfer and long-term retention.

===Studies===

Pilot Study: Free-form versus menu-based self-explanations
* Date: Fall 2008
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 65 students
* Total Participant Hours for the study: ~50-65 hours
* Data in the Data Shop: Yes
** [https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=307 Dataset: IWT Self-Explanation Study 0 (pilot) (Fall 2008) (tutors only)]
** '''Pre/Post Test Score Data:''' Yes
** '''Paper or Online Tests:''' Online
** '''Scanned Paper Tests:''' N/A
** '''Blank Tests:''' No
** '''Answer Keys: ''' No

Study 1: Self-Explanation Only versus Practice Only
* Date: Spring 2009
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 118 students
* Total Participant Hours for the study: ~100-120 hours
* Data in the Data Shop: Yes
** [https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=312 Dataset: IWT Self-Explanation Study 1 (Spring 2009) (tests only)]
**[https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=313 Dataset: IWT Self-Explanation Study 1 (Spring 2009) (tutors only)]
** '''Pre/Post Test Score Data:''' Yes
** '''Paper or Online Tests:''' Online
** '''Scanned Paper Tests:''' N/A
** '''Blank Tests:''' No
** '''Answer Keys: ''' No

Study 2: Analogy, Self-Explanation, and Practice
* Date: Fall 2009
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 99 students
* Total Participant Hours for the study: ~75-100 hours
* Data in the Data Shop: Yes
**[https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=328 Dataset: IWT Self-Explanation Study 2 (Fall 2009) (tests only)]
**[https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=372 Dataset: IWT Self-Explanation Study 2 (Fall 2009) (tutors only)]
** '''Pre/Post Test Score Data:''' Yes
** '''Paper or Online Tests:''' Online
** '''Scanned Paper Tests:''' N/A
** '''Blank Tests:''' No
** '''Answer Keys: ''' No

Study 3: Example Study, Self-Explanation, and Practice
* Date: Spring 2010
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 93 students
* Total Participant Hours for the study: ~80-95 hours
* Data in the Data Shop: Yes
**[https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=393 Dataset: IWT Self-Explanation Study 3 (Spring 2010) (tests only)]
**[https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=394 Dataset: IWT Self-Explanation Study 3 (Spring 2010) (tutors only)]
** '''Pre/Post Test Score Data:''' Yes
** '''Paper or Online Tests:''' Online
** '''Scanned Paper Tests:''' N/A
** '''Blank Tests:''' No
** '''Answer Keys: ''' No

Study 4: Adaptive Self-Explanation vs. Practice-Only
* Date: Fall 2010
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 142 students total (tutor data for 131 students)
* Total Participant Hours for the study: ~100 hours
* Data in the Data Shop: Yes

===Publications===

Wylie, R., Koedinger, K., and Mitamura T. (submitted) Testing the Generality and Efficiency of Self-Explanation in Second Language Learning.

Wylie, R., Sheng, M., Mitamura, T., and Koedinger, K. (submitted) Effects of Adapted Self-Explanation on Robust Learning of Second Language Grammar.

Wylie, R., Koedinger, K., and Mitamura, T. (2010) Extending the Self-Explanation Effect to Second Language Grammar Learning. International Conference of the Learning Sciences. Chicago, Illinois. June 29-July 2, 2010.

Wylie, R., Koedinger, K., and Mitamura, T. (2010) Analogies, Explanation, and Practice: Examining how task types affect second language grammar learning. Tenth International Conference on Intelligent Tutoring Systems. Pittsburgh, Pennsylvania. June 14-18, 2010.

Wylie, R., Koedinger, K., and Mitamura, T. (2009) Is Self-Explanation Always Better? The Effects of Adding Self-Explanation Prompts to an English Grammar Tutor. Cognitive Science. Amsterdam, The Netherlands. July 29 – August 1, 2009.

===Future Plans===

* Data Analysis for Study 5
* Cross-study data analysis
* Robust Learning Analysis
** Learning gains by Knowledge Component
** Long-term Retention Measure
** Transfer Measure (student writing samples)
* Paper on Efficiency Scores

[[Category:Data available in DataShop]]

Wylie - Intelligent Writing Tutor

2011-02-14T15:26:18Z

Ruth-Wylie: /* Studies */

== '''Project Title: Generalizing Self-Explanation''' ==
Report prepared for PSLC Site Visit, January 2011 
PSLC Research Thrust(s): Cognitive Factors 
PSLC Learnlab Course(s): English as a Second Language

{| class="wikitable" border=1
|-
| width="150pt" | PI: Ruth Wylie Department: HCII Organization: CMU
| width="150pt" | PI: Ken Koedinger Department: HCII Organization: CMU
| width="150pt" | PI: Teruko Mitamura Department: HCII Organization: CMU
|}

Others who have contributed 160 hours or more: 
PSLC Undergraduate Intern: Melissa Sheng, Rice University

{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''Study Name/Number''' || Pilot Study: Modalities of Self-Explanation || Study 1: Practice vs. Self-Explanation Only || Study 2: Analogy, Explanation, and Practice || Study 3: Worked Examples, Explanation, and Practice
|-
| '''Study Date''' || Fall 2008 || Spring 2009 || Fall 2009 || Spring 2010
|-
| '''LearnLab Site''' || University of Pittsburgh, ELI || University of Pittsburgh, ELI || University of Pittsburgh, ELI || University of Pittsburgh, ELI
|-
| '''LearnLab Course''' || ESL || ESL || ESL || ESL
|-
| '''Number of Students''' || 63 || 118 || 99 || 97
|-
|-
| '''DataShop''' ||
* '''Pre/Post Test Score Data:''' Yes
* '''Paper or Online Tests:''' Online
* '''Scanned Paper Tests:''' n/a
* '''Blank Tests:''' uploaded soon
* '''Answer Keys: ''' uploaded soon
||
* '''Pre/Post Test Score Data:''' Yes
* '''Paper or Online Tests:''' Online
* '''Scanned Paper Tests:''' n/a
* '''Blank Tests:''' uploaded soon
* '''Answer Keys: ''' uploaded soon
||
* '''Pre/Post Test Score Data:''' Yes
* '''Paper or Online Tests:''' Online
* '''Scanned Paper Tests:''' n/a
* '''Blank Tests:''' uploaded soon
* '''Answer Keys: ''' uploaded soon
||
* '''Pre/Post Test Score Data:''' Yes
* '''Paper or Online Tests:''' Online
* '''Scanned Paper Tests:''' n/a
* '''Blank Tests:''' uploaded soon
* '''Answer Keys: ''' uploaded soon
|}

{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''Study Name/Number''' || Study 4: Adaptive Self-Explanation Tutor
|-
| '''Study Date''' || Fall 2010
|-
| '''LearnLab Site''' || University of Pittsburgh, ELI
|-
| '''LearnLab Course''' || ESL
|-
| '''Number of Students''' || 131
|-
|-
| '''DataShop''' ||
* '''Pre/Post Test Score Data:''' Yes
* '''Paper or Online Tests:''' Online
* '''Scanned Paper Tests:''' n/a
* '''Blank Tests:''' uploaded soon
* '''Answer Keys: ''' uploaded soon
|}

== Project Plan Abstract ==
Prompting students to self-explain during problem solving has proven to be an effective instructional strategy across many domains. However, despite being called a “domain general” strategy, very little work has been done in areas outside of math and science. Thus, it remains an open question whether the self-explanation effect will hold in new and different domains like second language grammar learning. Through a series of classroom studies, we compare the effects of four tutoring systems: two that facilitate deep processing (prompted self-explanation and analogical comparison) and two that increase the rate of processing (example study and practice). Results show that while deep processing in general, and self-explanation specifically, is an effective strategy, it may not be an efficient approach compared to practice alone. These results suggest that the benefits of self-explanation over traditional practice may not be truly domain independent.

== Achievements ==
===Findings===
We have conducted four in-vivo studies with plans to do a fifth study during Fall 2010. The first study compared two different forms of self-explanation to a ecologically valid control condition of practice-only. We looked at both menu-based self-explanations (students chose the rule from a provided menu) and free-form responses (students typed their explanation into an empty text box). The second study compared self-explanation only to practice only in order to understand the effects that self-explanation alone had on student learning. Studies 3 and 4 introduced new instructional manipulations with goals to both increase learning and efficiency. Study 3 used analogical comparisons in an attempt to reduce the metalinguistic demands placed on the students, and Study 4 used example study in an attempt to reduce the amount of time students spent working with the tutor. The main finding from this series of studies is that while prompting students to self-explain does lead to significant learning gains, students who self-explain perform no better than students in a traditional practice condition. Furthermore, self-explanation requires significantly more time meaning it is inefficient compared to practice. These results differ from those found in previous self-explanation studies conducted largely in math and science domains.

{| class="wikitable"
|-
| [[Image:IWT_study1_learninggains.jpg]]
| [[Image:IWT_study2_learninggains.jpg]]
|-
| Fig. 1. Study 1 Learning Gains
| Fig. 2. Study 2 Learning Gains
|-
| [[Image:IWT_study3_learninggains.jpg]]
| [[Image:IWT_study4_learninggains.jpg]]
|-
| Fig. 3. Study 3 Learning Gains
| Fig. 4. Study 4 Learning Gains
|}
Figures 1-4: Across all four studies, students in all conditions show significant, but equal, learning gains. Students improve when prompted to self-explanation either alone (Study 2) or paired with tutored practice (Studies 1, 3 and 4), but there is no difference in learning between students who are prompted to self-explain and those that are not.

In addition to learning gain analysis, we have also compared efficiency scores across conditions for Studies 2, 3, and 4. Efficiency scores combine learning gains and time to complete instruction into a single measure. These results reveal a similar pattern: practice is much more efficient than self-explanation for learning the English article system.

{| class="wikitable"
|-
| [[Image:IWT_study2_efficiency.jpg]]
| [[Image:IWT_study3_efficiency.jpg]]
|-
| Fig. 5. Study 2 Efficiency Analysis
| Fig. 6. Study 3 Efficiency Analysis
|-
| [[Image:IWT_study4_efficiency.jpg]]
|-
| Fig. 7. Study 4 Learning Gains
|}
Figures 5-7: Efficiency Score Analysis. Across all studies, students in the practice-only condition were much more efficient than those in the self-explanation condition.

These studies address a key PSLC goal of identifying under what conditions instructional strategies lead to robust learning. Our results to date suggest that self-explanation may not be the best form of instruction for all domains. In addition to planning our final study, our current work investigates the effects of self-explanation prompts on robust learning measures like transfer and long-term retention.

===Studies===

Pilot Study: Free-form versus menu-based self-explanations
* Date: Fall 2008
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 65 students
* Total Participant Hours for the study: ~50-65 hours
* Data in the Data Shop: Yes
** [https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=307 Dataset: IWT Self-Explanation Study 0 (pilot) (Fall 2008) (tutors only)]
** '''Pre/Post Test Score Data:''' Yes
** '''Paper or Online Tests:''' Online
** '''Scanned Paper Tests:''' N/A
** '''Blank Tests:''' No
** '''Answer Keys: ''' No

Study 1: Self-Explanation Only versus Practice Only
* Date: Spring 2009
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 118 students
* Total Participant Hours for the study: ~100-120 hours
* Data in the Data Shop: Yes
** [https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=312 Dataset: IWT Self-Explanation Study 1 (Spring 2009) (tests only)]
**[https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=313 Dataset: IWT Self-Explanation Study 1 (Spring 2009) (tutors only)]
** '''Pre/Post Test Score Data:''' Yes
** '''Paper or Online Tests:''' Online
** '''Scanned Paper Tests:''' N/A
** '''Blank Tests:''' No
** '''Answer Keys: ''' No

Study 2: Analogy, Self-Explanation, and Practice
* Date: Fall 2009
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 99 students
* Total Participant Hours for the study: ~75-100 hours
* Data in the Data Shop: Yes
**[https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=328 Dataset: IWT Self-Explanation Study 2 (Fall 2009) (tests only)]
**[https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=372 Dataset: IWT Self-Explanation Study 2 (Fall 2009) (tutors only)]
** '''Pre/Post Test Score Data:''' Yes
** '''Paper or Online Tests:''' Online
** '''Scanned Paper Tests:''' N/A
** '''Blank Tests:''' No
** '''Answer Keys: ''' No

Study 3: Example Study, Self-Explanation, and Practice
* Date: Spring 2010
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 93 students
* Total Participant Hours for the study: ~80-95 hours
* Data in the Data Shop: Yes
**[https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=393 Dataset: IWT Self-Explanation Study 3 (Spring 2010) (tests only)]
**[https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=394 Dataset: IWT Self-Explanation Study 3 (Spring 2010) (tutors only)]
** '''Pre/Post Test Score Data:''' Yes
** '''Paper or Online Tests:''' Online
** '''Scanned Paper Tests:''' N/A
** '''Blank Tests:''' No
** '''Answer Keys: ''' No

Study 4: Adaptive Self-Explanation vs. Practice-Only
* Date: Fall 2010
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 142 students total (tutor data for 131 students)
* Total Participant Hours for the study: ~100 hours
* Data in the Data Shop: Yes

===Publications===

Wylie, R., Koedinger, K., and Mitamura, T. (2010) Extending the Self-Explanation Effect to Second Language Grammar Learning. International Conference of the Learning Sciences. Chicago, Illinois. June 29-July 2, 2010.

Wylie, R., Koedinger, K., and Mitamura, T. (2010) Analogies, Explanation, and Practice: Examining how task types affect second language grammar learning. Tenth International Conference on Intelligent Tutoring Systems. Pittsburgh, Pennsylvania. June 14-18, 2010.

Wylie, R., Koedinger, K., and Mitamura, T. (2009) Is Self-Explanation Always Better? The Effects of Adding Self-Explanation Prompts to an English Grammar Tutor. Cognitive Science. Amsterdam, The Netherlands. July 29 – August 1, 2009.

===Future Plans===

* Data Analysis for Study 5
* Cross-study data analysis
* Robust Learning Analysis
** Learning gains by Knowledge Component
** Long-term Retention Measure
** Transfer Measure (student writing samples)
* Paper on Efficiency Scores

[[Category:Data available in DataShop]]

Wylie - Intelligent Writing Tutor

2011-02-14T15:19:57Z

Ruth-Wylie: /* '''Project Title: Generalizing Self-Explanation''' */

== '''Project Title: Generalizing Self-Explanation''' ==
Report prepared for PSLC Site Visit, January 2011 
PSLC Research Thrust(s): Cognitive Factors 
PSLC Learnlab Course(s): English as a Second Language

{| class="wikitable" border=1
|-
| width="150pt" | PI: Ruth Wylie Department: HCII Organization: CMU
| width="150pt" | PI: Ken Koedinger Department: HCII Organization: CMU
| width="150pt" | PI: Teruko Mitamura Department: HCII Organization: CMU
|}

Others who have contributed 160 hours or more: 
PSLC Undergraduate Intern: Melissa Sheng, Rice University

{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''Study Name/Number''' || Pilot Study: Modalities of Self-Explanation || Study 1: Practice vs. Self-Explanation Only || Study 2: Analogy, Explanation, and Practice || Study 3: Worked Examples, Explanation, and Practice
|-
| '''Study Date''' || Fall 2008 || Spring 2009 || Fall 2009 || Spring 2010
|-
| '''LearnLab Site''' || University of Pittsburgh, ELI || University of Pittsburgh, ELI || University of Pittsburgh, ELI || University of Pittsburgh, ELI
|-
| '''LearnLab Course''' || ESL || ESL || ESL || ESL
|-
| '''Number of Students''' || 63 || 118 || 99 || 97
|-
|-
| '''DataShop''' ||
* '''Pre/Post Test Score Data:''' Yes
* '''Paper or Online Tests:''' Online
* '''Scanned Paper Tests:''' n/a
* '''Blank Tests:''' uploaded soon
* '''Answer Keys: ''' uploaded soon
||
* '''Pre/Post Test Score Data:''' Yes
* '''Paper or Online Tests:''' Online
* '''Scanned Paper Tests:''' n/a
* '''Blank Tests:''' uploaded soon
* '''Answer Keys: ''' uploaded soon
||
* '''Pre/Post Test Score Data:''' Yes
* '''Paper or Online Tests:''' Online
* '''Scanned Paper Tests:''' n/a
* '''Blank Tests:''' uploaded soon
* '''Answer Keys: ''' uploaded soon
||
* '''Pre/Post Test Score Data:''' Yes
* '''Paper or Online Tests:''' Online
* '''Scanned Paper Tests:''' n/a
* '''Blank Tests:''' uploaded soon
* '''Answer Keys: ''' uploaded soon
|}

{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''Study Name/Number''' || Study 4: Adaptive Self-Explanation Tutor
|-
| '''Study Date''' || Fall 2010
|-
| '''LearnLab Site''' || University of Pittsburgh, ELI
|-
| '''LearnLab Course''' || ESL
|-
| '''Number of Students''' || 131
|-
|-
| '''DataShop''' ||
* '''Pre/Post Test Score Data:''' Yes
* '''Paper or Online Tests:''' Online
* '''Scanned Paper Tests:''' n/a
* '''Blank Tests:''' uploaded soon
* '''Answer Keys: ''' uploaded soon
|}

== Project Plan Abstract ==
Prompting students to self-explain during problem solving has proven to be an effective instructional strategy across many domains. However, despite being called a “domain general” strategy, very little work has been done in areas outside of math and science. Thus, it remains an open question whether the self-explanation effect will hold in new and different domains like second language grammar learning. Through a series of classroom studies, we compare the effects of four tutoring systems: two that facilitate deep processing (prompted self-explanation and analogical comparison) and two that increase the rate of processing (example study and practice). Results show that while deep processing in general, and self-explanation specifically, is an effective strategy, it may not be an efficient approach compared to practice alone. These results suggest that the benefits of self-explanation over traditional practice may not be truly domain independent.

== Achievements ==
===Findings===
We have conducted four in-vivo studies with plans to do a fifth study during Fall 2010. The first study compared two different forms of self-explanation to a ecologically valid control condition of practice-only. We looked at both menu-based self-explanations (students chose the rule from a provided menu) and free-form responses (students typed their explanation into an empty text box). The second study compared self-explanation only to practice only in order to understand the effects that self-explanation alone had on student learning. Studies 3 and 4 introduced new instructional manipulations with goals to both increase learning and efficiency. Study 3 used analogical comparisons in an attempt to reduce the metalinguistic demands placed on the students, and Study 4 used example study in an attempt to reduce the amount of time students spent working with the tutor. The main finding from this series of studies is that while prompting students to self-explain does lead to significant learning gains, students who self-explain perform no better than students in a traditional practice condition. Furthermore, self-explanation requires significantly more time meaning it is inefficient compared to practice. These results differ from those found in previous self-explanation studies conducted largely in math and science domains.

{| class="wikitable"
|-
| [[Image:IWT_study1_learninggains.jpg]]
| [[Image:IWT_study2_learninggains.jpg]]
|-
| Fig. 1. Study 1 Learning Gains
| Fig. 2. Study 2 Learning Gains
|-
| [[Image:IWT_study3_learninggains.jpg]]
| [[Image:IWT_study4_learninggains.jpg]]
|-
| Fig. 3. Study 3 Learning Gains
| Fig. 4. Study 4 Learning Gains
|}
Figures 1-4: Across all four studies, students in all conditions show significant, but equal, learning gains. Students improve when prompted to self-explanation either alone (Study 2) or paired with tutored practice (Studies 1, 3 and 4), but there is no difference in learning between students who are prompted to self-explain and those that are not.

In addition to learning gain analysis, we have also compared efficiency scores across conditions for Studies 2, 3, and 4. Efficiency scores combine learning gains and time to complete instruction into a single measure. These results reveal a similar pattern: practice is much more efficient than self-explanation for learning the English article system.

{| class="wikitable"
|-
| [[Image:IWT_study2_efficiency.jpg]]
| [[Image:IWT_study3_efficiency.jpg]]
|-
| Fig. 5. Study 2 Efficiency Analysis
| Fig. 6. Study 3 Efficiency Analysis
|-
| [[Image:IWT_study4_efficiency.jpg]]
|-
| Fig. 7. Study 4 Learning Gains
|}
Figures 5-7: Efficiency Score Analysis. Across all studies, students in the practice-only condition were much more efficient than those in the self-explanation condition.

These studies address a key PSLC goal of identifying under what conditions instructional strategies lead to robust learning. Our results to date suggest that self-explanation may not be the best form of instruction for all domains. In addition to planning our final study, our current work investigates the effects of self-explanation prompts on robust learning measures like transfer and long-term retention.

===Studies===

Pilot Study: Free-form versus menu-based self-explanations
* Date: Fall 2008
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 65 students
* Total Participant Hours for the study: ~50-65 hours
* Data in the Data Shop: Yes
** [https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=307 Dataset: IWT Self-Explanation Study 0 (pilot) (Fall 2008) (tutors only)]
** '''Pre/Post Test Score Data:''' Yes
** '''Paper or Online Tests:''' Online
** '''Scanned Paper Tests:''' N/A
** '''Blank Tests:''' No
** '''Answer Keys: ''' No

Study 1: Self-Explanation Only versus Practice Only
* Date: Spring 2009
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 118 students
* Total Participant Hours for the study: ~100-120 hours
* Data in the Data Shop: Yes
** [https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=312 Dataset: IWT Self-Explanation Study 1 (Spring 2009) (tests only)]
**[https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=313 Dataset: IWT Self-Explanation Study 1 (Spring 2009) (tutors only)]
** '''Pre/Post Test Score Data:''' Yes
** '''Paper or Online Tests:''' Online
** '''Scanned Paper Tests:''' N/A
** '''Blank Tests:''' No
** '''Answer Keys: ''' No

Study 2: Analogy, Self-Explanation, and Practice
* Date: Fall 2009
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 99 students
* Total Participant Hours for the study: ~75-100 hours
* Data in the Data Shop: Yes
**[https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=328 Dataset: IWT Self-Explanation Study 2 (Fall 2009) (tests only)]
**[https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=372 Dataset: IWT Self-Explanation Study 2 (Fall 2009) (tutors only)]
** '''Pre/Post Test Score Data:''' Yes
** '''Paper or Online Tests:''' Online
** '''Scanned Paper Tests:''' N/A
** '''Blank Tests:''' No
** '''Answer Keys: ''' No

Study 3: Example Study, Self-Explanation, and Practice
* Date: Spring 2010
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 93 students
* Total Participant Hours for the study: ~80-95 hours
* Data in the Data Shop: Yes
**[https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=393 Dataset: IWT Self-Explanation Study 3 (Spring 2010) (tests only)]
**[https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=394 Dataset: IWT Self-Explanation Study 3 (Spring 2010) (tutors only)]
** '''Pre/Post Test Score Data:''' Yes
** '''Paper or Online Tests:''' Online
** '''Scanned Paper Tests:''' N/A
** '''Blank Tests:''' No
** '''Answer Keys: ''' No

Study 4: Adaptive Self-Explanation vs. Practice-Only
* Date: Fall 2010
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 142 students
* Total Participant Hours for the study: ~100 hours
* Data in the Data Shop: Yes

===Publications===

Wylie, R., Koedinger, K., and Mitamura, T. (2010) Extending the Self-Explanation Effect to Second Language Grammar Learning. International Conference of the Learning Sciences. Chicago, Illinois. June 29-July 2, 2010.

Wylie, R., Koedinger, K., and Mitamura, T. (2010) Analogies, Explanation, and Practice: Examining how task types affect second language grammar learning. Tenth International Conference on Intelligent Tutoring Systems. Pittsburgh, Pennsylvania. June 14-18, 2010.

Wylie, R., Koedinger, K., and Mitamura, T. (2009) Is Self-Explanation Always Better? The Effects of Adding Self-Explanation Prompts to an English Grammar Tutor. Cognitive Science. Amsterdam, The Netherlands. July 29 – August 1, 2009.

===Future Plans===

* Data Analysis for Study 5
* Cross-study data analysis
* Robust Learning Analysis
** Learning gains by Knowledge Component
** Long-term Retention Measure
** Transfer Measure (student writing samples)
* Paper on Efficiency Scores

[[Category:Data available in DataShop]]

Wylie - Intelligent Writing Tutor

2011-02-14T15:14:19Z

Ruth-Wylie: /* II. Studies */

== '''Project Title: Generalizing Self-Explanation''' ==
Report prepared for PSLC Site Visit, January 2011 
PSLC Research Thrust(s): Cognitive Factors 
PSLC Learnlab Course(s): English as a Second Language

{| class="wikitable" border=1
|-
| width="150pt" | PI: Ruth Wylie Department: HCII Organization: CMU
| width="150pt" | PI: Ken Koedinger Department: HCII Organization: CMU
| width="150pt" | PI: Teruko Mitamura Department: HCII Organization: CMU
|}

Others who have contributed 160 hours or more: 
PSLC Undergraduate Intern: Melissa Sheng, Rice University

{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''Study Name/Number''' || Pilot Study: Modalities of Self-Explanation || Study 1: Practice vs. Self-Explanation Only || Study 2: Analogy, Explanation, and Practice || Study 3: Worked Examples, Explanation, and Practice
|-
| '''Study Date''' || Fall 2008 || Spring 2009 || Fall 2009 || Spring 2010
|-
| '''LearnLab Site''' || University of Pittsburgh, ELI || University of Pittsburgh, ELI || University of Pittsburgh, ELI || University of Pittsburgh, ELI
|-
| '''LearnLab Course''' || ESL || ESL || ESL || ESL
|-
| '''Number of Students''' || 63 || 118 || 99 || 97
|-
|-
| '''DataShop''' ||
* '''Pre/Post Test Score Data:''' Yes
* '''Paper or Online Tests:''' Online
* '''Scanned Paper Tests:''' n/a
* '''Blank Tests:''' uploaded soon
* '''Answer Keys: ''' uploaded soon
||
* '''Pre/Post Test Score Data:''' Yes
* '''Paper or Online Tests:''' Online
* '''Scanned Paper Tests:''' n/a
* '''Blank Tests:''' uploaded soon
* '''Answer Keys: ''' uploaded soon
||
* '''Pre/Post Test Score Data:''' Yes
* '''Paper or Online Tests:''' Online
* '''Scanned Paper Tests:''' n/a
* '''Blank Tests:''' uploaded soon
* '''Answer Keys: ''' uploaded soon
||
* '''Pre/Post Test Score Data:''' Yes
* '''Paper or Online Tests:''' Online
* '''Scanned Paper Tests:''' n/a
* '''Blank Tests:''' uploaded soon
* '''Answer Keys: ''' uploaded soon
|}

== Project Plan Abstract ==
Prompting students to self-explain during problem solving has proven to be an effective instructional strategy across many domains. However, despite being called a “domain general” strategy, very little work has been done in areas outside of math and science. Thus, it remains an open question whether the self-explanation effect will hold in new and different domains like second language grammar learning. Through a series of classroom studies, we compare the effects of four tutoring systems: two that facilitate deep processing (prompted self-explanation and analogical comparison) and two that increase the rate of processing (example study and practice). Results show that while deep processing in general, and self-explanation specifically, is an effective strategy, it may not be an efficient approach compared to practice alone. These results suggest that the benefits of self-explanation over traditional practice may not be truly domain independent.

== Achievements ==
===Findings===
We have conducted four in-vivo studies with plans to do a fifth study during Fall 2010. The first study compared two different forms of self-explanation to a ecologically valid control condition of practice-only. We looked at both menu-based self-explanations (students chose the rule from a provided menu) and free-form responses (students typed their explanation into an empty text box). The second study compared self-explanation only to practice only in order to understand the effects that self-explanation alone had on student learning. Studies 3 and 4 introduced new instructional manipulations with goals to both increase learning and efficiency. Study 3 used analogical comparisons in an attempt to reduce the metalinguistic demands placed on the students, and Study 4 used example study in an attempt to reduce the amount of time students spent working with the tutor. The main finding from this series of studies is that while prompting students to self-explain does lead to significant learning gains, students who self-explain perform no better than students in a traditional practice condition. Furthermore, self-explanation requires significantly more time meaning it is inefficient compared to practice. These results differ from those found in previous self-explanation studies conducted largely in math and science domains.

{| class="wikitable"
|-
| [[Image:IWT_study1_learninggains.jpg]]
| [[Image:IWT_study2_learninggains.jpg]]
|-
| Fig. 1. Study 1 Learning Gains
| Fig. 2. Study 2 Learning Gains
|-
| [[Image:IWT_study3_learninggains.jpg]]
| [[Image:IWT_study4_learninggains.jpg]]
|-
| Fig. 3. Study 3 Learning Gains
| Fig. 4. Study 4 Learning Gains
|}
Figures 1-4: Across all four studies, students in all conditions show significant, but equal, learning gains. Students improve when prompted to self-explanation either alone (Study 2) or paired with tutored practice (Studies 1, 3 and 4), but there is no difference in learning between students who are prompted to self-explain and those that are not.

In addition to learning gain analysis, we have also compared efficiency scores across conditions for Studies 2, 3, and 4. Efficiency scores combine learning gains and time to complete instruction into a single measure. These results reveal a similar pattern: practice is much more efficient than self-explanation for learning the English article system.

{| class="wikitable"
|-
| [[Image:IWT_study2_efficiency.jpg]]
| [[Image:IWT_study3_efficiency.jpg]]
|-
| Fig. 5. Study 2 Efficiency Analysis
| Fig. 6. Study 3 Efficiency Analysis
|-
| [[Image:IWT_study4_efficiency.jpg]]
|-
| Fig. 7. Study 4 Learning Gains
|}
Figures 5-7: Efficiency Score Analysis. Across all studies, students in the practice-only condition were much more efficient than those in the self-explanation condition.

These studies address a key PSLC goal of identifying under what conditions instructional strategies lead to robust learning. Our results to date suggest that self-explanation may not be the best form of instruction for all domains. In addition to planning our final study, our current work investigates the effects of self-explanation prompts on robust learning measures like transfer and long-term retention.

===Studies===

Pilot Study: Free-form versus menu-based self-explanations
* Date: Fall 2008
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 65 students
* Total Participant Hours for the study: ~50-65 hours
* Data in the Data Shop: Yes
** [https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=307 Dataset: IWT Self-Explanation Study 0 (pilot) (Fall 2008) (tutors only)]
** '''Pre/Post Test Score Data:''' Yes
** '''Paper or Online Tests:''' Online
** '''Scanned Paper Tests:''' N/A
** '''Blank Tests:''' No
** '''Answer Keys: ''' No

Study 1: Self-Explanation Only versus Practice Only
* Date: Spring 2009
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 118 students
* Total Participant Hours for the study: ~100-120 hours
* Data in the Data Shop: Yes
** [https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=312 Dataset: IWT Self-Explanation Study 1 (Spring 2009) (tests only)]
**[https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=313 Dataset: IWT Self-Explanation Study 1 (Spring 2009) (tutors only)]
** '''Pre/Post Test Score Data:''' Yes
** '''Paper or Online Tests:''' Online
** '''Scanned Paper Tests:''' N/A
** '''Blank Tests:''' No
** '''Answer Keys: ''' No

Study 2: Analogy, Self-Explanation, and Practice
* Date: Fall 2009
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 99 students
* Total Participant Hours for the study: ~75-100 hours
* Data in the Data Shop: Yes
**[https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=328 Dataset: IWT Self-Explanation Study 2 (Fall 2009) (tests only)]
**[https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=372 Dataset: IWT Self-Explanation Study 2 (Fall 2009) (tutors only)]
** '''Pre/Post Test Score Data:''' Yes
** '''Paper or Online Tests:''' Online
** '''Scanned Paper Tests:''' N/A
** '''Blank Tests:''' No
** '''Answer Keys: ''' No

Study 3: Example Study, Self-Explanation, and Practice
* Date: Spring 2010
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 93 students
* Total Participant Hours for the study: ~80-95 hours
* Data in the Data Shop: Yes
**[https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=393 Dataset: IWT Self-Explanation Study 3 (Spring 2010) (tests only)]
**[https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=394 Dataset: IWT Self-Explanation Study 3 (Spring 2010) (tutors only)]
** '''Pre/Post Test Score Data:''' Yes
** '''Paper or Online Tests:''' Online
** '''Scanned Paper Tests:''' N/A
** '''Blank Tests:''' No
** '''Answer Keys: ''' No

Study 4: Adaptive Self-Explanation vs. Practice-Only
* Date: Fall 2010
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 142 students
* Total Participant Hours for the study: ~100 hours
* Data in the Data Shop: Yes

===Publications===

Wylie, R., Koedinger, K., and Mitamura, T. (2010) Extending the Self-Explanation Effect to Second Language Grammar Learning. International Conference of the Learning Sciences. Chicago, Illinois. June 29-July 2, 2010.

Wylie, R., Koedinger, K., and Mitamura, T. (2010) Analogies, Explanation, and Practice: Examining how task types affect second language grammar learning. Tenth International Conference on Intelligent Tutoring Systems. Pittsburgh, Pennsylvania. June 14-18, 2010.

Wylie, R., Koedinger, K., and Mitamura, T. (2009) Is Self-Explanation Always Better? The Effects of Adding Self-Explanation Prompts to an English Grammar Tutor. Cognitive Science. Amsterdam, The Netherlands. July 29 – August 1, 2009.

===Future Plans===

* Data Analysis for Study 5
* Cross-study data analysis
* Robust Learning Analysis
** Learning gains by Knowledge Component
** Long-term Retention Measure
** Transfer Measure (student writing samples)
* Paper on Efficiency Scores

[[Category:Data available in DataShop]]

Wylie - Intelligent Writing Tutor

2011-02-14T15:13:56Z

Ruth-Wylie: /* I. Findings */

== '''Project Title: Generalizing Self-Explanation''' ==
Report prepared for PSLC Site Visit, January 2011 
PSLC Research Thrust(s): Cognitive Factors 
PSLC Learnlab Course(s): English as a Second Language

{| class="wikitable" border=1
|-
| width="150pt" | PI: Ruth Wylie Department: HCII Organization: CMU
| width="150pt" | PI: Ken Koedinger Department: HCII Organization: CMU
| width="150pt" | PI: Teruko Mitamura Department: HCII Organization: CMU
|}

Others who have contributed 160 hours or more: 
PSLC Undergraduate Intern: Melissa Sheng, Rice University

{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''Study Name/Number''' || Pilot Study: Modalities of Self-Explanation || Study 1: Practice vs. Self-Explanation Only || Study 2: Analogy, Explanation, and Practice || Study 3: Worked Examples, Explanation, and Practice
|-
| '''Study Date''' || Fall 2008 || Spring 2009 || Fall 2009 || Spring 2010
|-
| '''LearnLab Site''' || University of Pittsburgh, ELI || University of Pittsburgh, ELI || University of Pittsburgh, ELI || University of Pittsburgh, ELI
|-
| '''LearnLab Course''' || ESL || ESL || ESL || ESL
|-
| '''Number of Students''' || 63 || 118 || 99 || 97
|-
|-
| '''DataShop''' ||
* '''Pre/Post Test Score Data:''' Yes
* '''Paper or Online Tests:''' Online
* '''Scanned Paper Tests:''' n/a
* '''Blank Tests:''' uploaded soon
* '''Answer Keys: ''' uploaded soon
||
* '''Pre/Post Test Score Data:''' Yes
* '''Paper or Online Tests:''' Online
* '''Scanned Paper Tests:''' n/a
* '''Blank Tests:''' uploaded soon
* '''Answer Keys: ''' uploaded soon
||
* '''Pre/Post Test Score Data:''' Yes
* '''Paper or Online Tests:''' Online
* '''Scanned Paper Tests:''' n/a
* '''Blank Tests:''' uploaded soon
* '''Answer Keys: ''' uploaded soon
||
* '''Pre/Post Test Score Data:''' Yes
* '''Paper or Online Tests:''' Online
* '''Scanned Paper Tests:''' n/a
* '''Blank Tests:''' uploaded soon
* '''Answer Keys: ''' uploaded soon
|}

== Project Plan Abstract ==
Prompting students to self-explain during problem solving has proven to be an effective instructional strategy across many domains. However, despite being called a “domain general” strategy, very little work has been done in areas outside of math and science. Thus, it remains an open question whether the self-explanation effect will hold in new and different domains like second language grammar learning. Through a series of classroom studies, we compare the effects of four tutoring systems: two that facilitate deep processing (prompted self-explanation and analogical comparison) and two that increase the rate of processing (example study and practice). Results show that while deep processing in general, and self-explanation specifically, is an effective strategy, it may not be an efficient approach compared to practice alone. These results suggest that the benefits of self-explanation over traditional practice may not be truly domain independent.

== Achievements ==
===Findings===
We have conducted four in-vivo studies with plans to do a fifth study during Fall 2010. The first study compared two different forms of self-explanation to a ecologically valid control condition of practice-only. We looked at both menu-based self-explanations (students chose the rule from a provided menu) and free-form responses (students typed their explanation into an empty text box). The second study compared self-explanation only to practice only in order to understand the effects that self-explanation alone had on student learning. Studies 3 and 4 introduced new instructional manipulations with goals to both increase learning and efficiency. Study 3 used analogical comparisons in an attempt to reduce the metalinguistic demands placed on the students, and Study 4 used example study in an attempt to reduce the amount of time students spent working with the tutor. The main finding from this series of studies is that while prompting students to self-explain does lead to significant learning gains, students who self-explain perform no better than students in a traditional practice condition. Furthermore, self-explanation requires significantly more time meaning it is inefficient compared to practice. These results differ from those found in previous self-explanation studies conducted largely in math and science domains.

{| class="wikitable"
|-
| [[Image:IWT_study1_learninggains.jpg]]
| [[Image:IWT_study2_learninggains.jpg]]
|-
| Fig. 1. Study 1 Learning Gains
| Fig. 2. Study 2 Learning Gains
|-
| [[Image:IWT_study3_learninggains.jpg]]
| [[Image:IWT_study4_learninggains.jpg]]
|-
| Fig. 3. Study 3 Learning Gains
| Fig. 4. Study 4 Learning Gains
|}
Figures 1-4: Across all four studies, students in all conditions show significant, but equal, learning gains. Students improve when prompted to self-explanation either alone (Study 2) or paired with tutored practice (Studies 1, 3 and 4), but there is no difference in learning between students who are prompted to self-explain and those that are not.

In addition to learning gain analysis, we have also compared efficiency scores across conditions for Studies 2, 3, and 4. Efficiency scores combine learning gains and time to complete instruction into a single measure. These results reveal a similar pattern: practice is much more efficient than self-explanation for learning the English article system.

{| class="wikitable"
|-
| [[Image:IWT_study2_efficiency.jpg]]
| [[Image:IWT_study3_efficiency.jpg]]
|-
| Fig. 5. Study 2 Efficiency Analysis
| Fig. 6. Study 3 Efficiency Analysis
|-
| [[Image:IWT_study4_efficiency.jpg]]
|-
| Fig. 7. Study 4 Learning Gains
|}
Figures 5-7: Efficiency Score Analysis. Across all studies, students in the practice-only condition were much more efficient than those in the self-explanation condition.

These studies address a key PSLC goal of identifying under what conditions instructional strategies lead to robust learning. Our results to date suggest that self-explanation may not be the best form of instruction for all domains. In addition to planning our final study, our current work investigates the effects of self-explanation prompts on robust learning measures like transfer and long-term retention.

===II. Studies===

Pilot Study: Free-form versus menu-based self-explanations
* Date: Fall 2008
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 65 students
* Total Participant Hours for the study: ~50-65 hours
* Data in the Data Shop: Yes
** [https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=307 Dataset: IWT Self-Explanation Study 0 (pilot) (Fall 2008) (tutors only)]
** '''Pre/Post Test Score Data:''' Yes
** '''Paper or Online Tests:''' Online
** '''Scanned Paper Tests:''' N/A
** '''Blank Tests:''' No
** '''Answer Keys: ''' No

Study 1: Self-Explanation Only versus Practice Only
* Date: Spring 2009
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 118 students
* Total Participant Hours for the study: ~100-120 hours
* Data in the Data Shop: Yes
** [https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=312 Dataset: IWT Self-Explanation Study 1 (Spring 2009) (tests only)]
**[https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=313 Dataset: IWT Self-Explanation Study 1 (Spring 2009) (tutors only)]
** '''Pre/Post Test Score Data:''' Yes
** '''Paper or Online Tests:''' Online
** '''Scanned Paper Tests:''' N/A
** '''Blank Tests:''' No
** '''Answer Keys: ''' No

Study 2: Analogy, Self-Explanation, and Practice
* Date: Fall 2009
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 99 students
* Total Participant Hours for the study: ~75-100 hours
* Data in the Data Shop: Yes
**[https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=328 Dataset: IWT Self-Explanation Study 2 (Fall 2009) (tests only)]
**[https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=372 Dataset: IWT Self-Explanation Study 2 (Fall 2009) (tutors only)]
** '''Pre/Post Test Score Data:''' Yes
** '''Paper or Online Tests:''' Online
** '''Scanned Paper Tests:''' N/A
** '''Blank Tests:''' No
** '''Answer Keys: ''' No

Study 3: Example Study, Self-Explanation, and Practice
* Date: Spring 2010
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 93 students
* Total Participant Hours for the study: ~80-95 hours
* Data in the Data Shop: Yes
**[https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=393 Dataset: IWT Self-Explanation Study 3 (Spring 2010) (tests only)]
**[https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=394 Dataset: IWT Self-Explanation Study 3 (Spring 2010) (tutors only)]
** '''Pre/Post Test Score Data:''' Yes
** '''Paper or Online Tests:''' Online
** '''Scanned Paper Tests:''' N/A
** '''Blank Tests:''' No
** '''Answer Keys: ''' No

Study 4: Adaptive Self-Explanation vs. Practice-Only
* Date: Fall 2010
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 142 students
* Total Participant Hours for the study: ~100 hours
* Data in the Data Shop: Yes

===Publications===

Wylie, R., Koedinger, K., and Mitamura, T. (2010) Extending the Self-Explanation Effect to Second Language Grammar Learning. International Conference of the Learning Sciences. Chicago, Illinois. June 29-July 2, 2010.

Wylie, R., Koedinger, K., and Mitamura, T. (2010) Analogies, Explanation, and Practice: Examining how task types affect second language grammar learning. Tenth International Conference on Intelligent Tutoring Systems. Pittsburgh, Pennsylvania. June 14-18, 2010.

Wylie, R., Koedinger, K., and Mitamura, T. (2009) Is Self-Explanation Always Better? The Effects of Adding Self-Explanation Prompts to an English Grammar Tutor. Cognitive Science. Amsterdam, The Netherlands. July 29 – August 1, 2009.

===Future Plans===

* Data Analysis for Study 5
* Cross-study data analysis
* Robust Learning Analysis
** Learning gains by Knowledge Component
** Long-term Retention Measure
** Transfer Measure (student writing samples)
* Paper on Efficiency Scores

[[Category:Data available in DataShop]]

Wylie - Intelligent Writing Tutor

2011-01-24T00:19:57Z

Ruth-Wylie: /* II. Studies */

== '''Project Title: Generalizing Self-Explanation''' ==
Report prepared for PSLC Site Visit, January 2011 
PSLC Research Thrust(s): Cognitive Factors 
PSLC Learnlab Course(s): English as a Second Language

{| class="wikitable" border=1
|-
| width="150pt" | PI: Ruth Wylie Department: HCII Organization: CMU
| width="150pt" | PI: Ken Koedinger Department: HCII Organization: CMU
| width="150pt" | PI: Teruko Mitamura Department: HCII Organization: CMU
|}

Others who have contributed 160 hours or more: 
PSLC Undergraduate Intern: Melissa Sheng, Rice University

{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''Study Name/Number''' || Pilot Study: Modalities of Self-Explanation || Study 1: Practice vs. Self-Explanation Only || Study 2: Analogy, Explanation, and Practice || Study 3: Worked Examples, Explanation, and Practice
|-
| '''Study Date''' || Fall 2008 || Spring 2009 || Fall 2009 || Spring 2010
|-
| '''LearnLab Site''' || University of Pittsburgh, ELI || University of Pittsburgh, ELI || University of Pittsburgh, ELI || University of Pittsburgh, ELI
|-
| '''LearnLab Course''' || ESL || ESL || ESL || ESL
|-
| '''Number of Students''' || 63 || 118 || 99 || 97
|-
|-
| '''DataShop''' ||
* '''Pre/Post Test Score Data:''' Yes
* '''Paper or Online Tests:''' Online
* '''Scanned Paper Tests:''' n/a
* '''Blank Tests:''' uploaded soon
* '''Answer Keys: ''' uploaded soon
||
* '''Pre/Post Test Score Data:''' Yes
* '''Paper or Online Tests:''' Online
* '''Scanned Paper Tests:''' n/a
* '''Blank Tests:''' uploaded soon
* '''Answer Keys: ''' uploaded soon
||
* '''Pre/Post Test Score Data:''' Yes
* '''Paper or Online Tests:''' Online
* '''Scanned Paper Tests:''' n/a
* '''Blank Tests:''' uploaded soon
* '''Answer Keys: ''' uploaded soon
||
* '''Pre/Post Test Score Data:''' Yes
* '''Paper or Online Tests:''' Online
* '''Scanned Paper Tests:''' n/a
* '''Blank Tests:''' uploaded soon
* '''Answer Keys: ''' uploaded soon
|}

== Project Plan Abstract ==
Prompting students to self-explain during problem solving has proven to be an effective instructional strategy across many domains. However, despite being called a “domain general” strategy, very little work has been done in areas outside of math and science. Thus, it remains an open question whether the self-explanation effect will hold in new and different domains like second language grammar learning. Through a series of classroom studies, we compare the effects of four tutoring systems: two that facilitate deep processing (prompted self-explanation and analogical comparison) and two that increase the rate of processing (example study and practice). Results show that while deep processing in general, and self-explanation specifically, is an effective strategy, it may not be an efficient approach compared to practice alone. These results suggest that the benefits of self-explanation over traditional practice may not be truly domain independent.

== Achievements ==
===I. Findings===
We have conducted four in-vivo studies with plans to do a fifth study during Fall 2010. The first study compared two different forms of self-explanation to a ecologically valid control condition of practice-only. We looked at both menu-based self-explanations (students chose the rule from a provided menu) and free-form responses (students typed their explanation into an empty text box). The second study compared self-explanation only to practice only in order to understand the effects that self-explanation alone had on student learning. Studies 3 and 4 introduced new instructional manipulations with goals to both increase learning and efficiency. Study 3 used analogical comparisons in an attempt to reduce the metalinguistic demands placed on the students, and Study 4 used example study in an attempt to reduce the amount of time students spent working with the tutor. The main finding from this series of studies is that while prompting students to self-explain does lead to significant learning gains, students who self-explain perform no better than students in a traditional practice condition. Furthermore, self-explanation requires significantly more time meaning it is inefficient compared to practice. These results differ from those found in previous self-explanation studies conducted largely in math and science domains.

{| class="wikitable"
|-
| [[Image:IWT_study1_learninggains.jpg]]
| [[Image:IWT_study2_learninggains.jpg]]
|-
| Fig. 1. Study 1 Learning Gains
| Fig. 2. Study 2 Learning Gains
|-
| [[Image:IWT_study3_learninggains.jpg]]
| [[Image:IWT_study4_learninggains.jpg]]
|-
| Fig. 3. Study 3 Learning Gains
| Fig. 4. Study 4 Learning Gains
|}
Figures 1-4: Across all four studies, students in all conditions show significant, but equal, learning gains. Students improve when prompted to self-explanation either alone (Study 2) or paired with tutored practice (Studies 1, 3 and 4), but there is no difference in learning between students who are prompted to self-explain and those that are not.

In addition to learning gain analysis, we have also compared efficiency scores across conditions for Studies 2, 3, and 4. Efficiency scores combine learning gains and time to complete instruction into a single measure. These results reveal a similar pattern: practice is much more efficient than self-explanation for learning the English article system.

{| class="wikitable"
|-
| [[Image:IWT_study2_efficiency.jpg]]
| [[Image:IWT_study3_efficiency.jpg]]
|-
| Fig. 5. Study 2 Efficiency Analysis
| Fig. 6. Study 3 Efficiency Analysis
|-
| [[Image:IWT_study4_efficiency.jpg]]
|-
| Fig. 7. Study 4 Learning Gains
|}
Figures 5-7: Efficiency Score Analysis. Across all studies, students in the practice-only condition were much more efficient than those in the self-explanation condition.

These studies address a key PSLC goal of identifying under what conditions instructional strategies lead to robust learning. Our results to date suggest that self-explanation may not be the best form of instruction for all domains. In addition to planning our final study, our current work investigates the effects of self-explanation prompts on robust learning measures like transfer and long-term retention.

===II. Studies===

Pilot Study: Free-form versus menu-based self-explanations
* Date: Fall 2008
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 65 students
* Total Participant Hours for the study: ~50-65 hours
* Data in the Data Shop: Yes
** [https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=307 Dataset: IWT Self-Explanation Study 0 (pilot) (Fall 2008) (tutors only)]
** '''Pre/Post Test Score Data:''' Yes
** '''Paper or Online Tests:''' Online
** '''Scanned Paper Tests:''' N/A
** '''Blank Tests:''' No
** '''Answer Keys: ''' No

Study 1: Self-Explanation Only versus Practice Only
* Date: Spring 2009
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 118 students
* Total Participant Hours for the study: ~100-120 hours
* Data in the Data Shop: Yes
** [https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=312 Dataset: IWT Self-Explanation Study 1 (Spring 2009) (tests only)]
**[https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=313 Dataset: IWT Self-Explanation Study 1 (Spring 2009) (tutors only)]
** '''Pre/Post Test Score Data:''' Yes
** '''Paper or Online Tests:''' Online
** '''Scanned Paper Tests:''' N/A
** '''Blank Tests:''' No
** '''Answer Keys: ''' No

Study 2: Analogy, Self-Explanation, and Practice
* Date: Fall 2009
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 99 students
* Total Participant Hours for the study: ~75-100 hours
* Data in the Data Shop: Yes
**[https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=328 Dataset: IWT Self-Explanation Study 2 (Fall 2009) (tests only)]
**[https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=372 Dataset: IWT Self-Explanation Study 2 (Fall 2009) (tutors only)]
** '''Pre/Post Test Score Data:''' Yes
** '''Paper or Online Tests:''' Online
** '''Scanned Paper Tests:''' N/A
** '''Blank Tests:''' No
** '''Answer Keys: ''' No

Study 3: Example Study, Self-Explanation, and Practice
* Date: Spring 2010
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 93 students
* Total Participant Hours for the study: ~80-95 hours
* Data in the Data Shop: Yes
**[https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=393 Dataset: IWT Self-Explanation Study 3 (Spring 2010) (tests only)]
**[https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=394 Dataset: IWT Self-Explanation Study 3 (Spring 2010) (tutors only)]
** '''Pre/Post Test Score Data:''' Yes
** '''Paper or Online Tests:''' Online
** '''Scanned Paper Tests:''' N/A
** '''Blank Tests:''' No
** '''Answer Keys: ''' No

Study 4: Adaptive Self-Explanation vs. Practice-Only
* Date: Fall 2010
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 142 students
* Total Participant Hours for the study: ~100 hours
* Data in the Data Shop: Yes

===Publications===

Wylie, R., Koedinger, K., and Mitamura, T. (2010) Extending the Self-Explanation Effect to Second Language Grammar Learning. International Conference of the Learning Sciences. Chicago, Illinois. June 29-July 2, 2010.

Wylie, R., Koedinger, K., and Mitamura, T. (2010) Analogies, Explanation, and Practice: Examining how task types affect second language grammar learning. Tenth International Conference on Intelligent Tutoring Systems. Pittsburgh, Pennsylvania. June 14-18, 2010.

Wylie, R., Koedinger, K., and Mitamura, T. (2009) Is Self-Explanation Always Better? The Effects of Adding Self-Explanation Prompts to an English Grammar Tutor. Cognitive Science. Amsterdam, The Netherlands. July 29 – August 1, 2009.

===Future Plans===

* Data Analysis for Study 5
* Cross-study data analysis
* Robust Learning Analysis
** Learning gains by Knowledge Component
** Long-term Retention Measure
** Transfer Measure (student writing samples)
* Paper on Efficiency Scores

[[Category:Data available in DataShop]]

Wylie - Intelligent Writing Tutor

2011-01-24T00:13:02Z

Ruth-Wylie: /* '''Project Title: Generalizing Self-Explanation''' */

== '''Project Title: Generalizing Self-Explanation''' ==
Report prepared for PSLC Site Visit, January 2011 
PSLC Research Thrust(s): Cognitive Factors 
PSLC Learnlab Course(s): English as a Second Language

{| class="wikitable" border=1
|-
| width="150pt" | PI: Ruth Wylie Department: HCII Organization: CMU
| width="150pt" | PI: Ken Koedinger Department: HCII Organization: CMU
| width="150pt" | PI: Teruko Mitamura Department: HCII Organization: CMU
|}

Others who have contributed 160 hours or more: 
PSLC Undergraduate Intern: Melissa Sheng, Rice University

{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''Study Name/Number''' || Pilot Study: Modalities of Self-Explanation || Study 1: Practice vs. Self-Explanation Only || Study 2: Analogy, Explanation, and Practice || Study 3: Worked Examples, Explanation, and Practice
|-
| '''Study Date''' || Fall 2008 || Spring 2009 || Fall 2009 || Spring 2010
|-
| '''LearnLab Site''' || University of Pittsburgh, ELI || University of Pittsburgh, ELI || University of Pittsburgh, ELI || University of Pittsburgh, ELI
|-
| '''LearnLab Course''' || ESL || ESL || ESL || ESL
|-
| '''Number of Students''' || 63 || 118 || 99 || 97
|-
|-
| '''DataShop''' ||
* '''Pre/Post Test Score Data:''' Yes
* '''Paper or Online Tests:''' Online
* '''Scanned Paper Tests:''' n/a
* '''Blank Tests:''' uploaded soon
* '''Answer Keys: ''' uploaded soon
||
* '''Pre/Post Test Score Data:''' Yes
* '''Paper or Online Tests:''' Online
* '''Scanned Paper Tests:''' n/a
* '''Blank Tests:''' uploaded soon
* '''Answer Keys: ''' uploaded soon
||
* '''Pre/Post Test Score Data:''' Yes
* '''Paper or Online Tests:''' Online
* '''Scanned Paper Tests:''' n/a
* '''Blank Tests:''' uploaded soon
* '''Answer Keys: ''' uploaded soon
||
* '''Pre/Post Test Score Data:''' Yes
* '''Paper or Online Tests:''' Online
* '''Scanned Paper Tests:''' n/a
* '''Blank Tests:''' uploaded soon
* '''Answer Keys: ''' uploaded soon
|}

== Project Plan Abstract ==
Prompting students to self-explain during problem solving has proven to be an effective instructional strategy across many domains. However, despite being called a “domain general” strategy, very little work has been done in areas outside of math and science. Thus, it remains an open question whether the self-explanation effect will hold in new and different domains like second language grammar learning. Through a series of classroom studies, we compare the effects of four tutoring systems: two that facilitate deep processing (prompted self-explanation and analogical comparison) and two that increase the rate of processing (example study and practice). Results show that while deep processing in general, and self-explanation specifically, is an effective strategy, it may not be an efficient approach compared to practice alone. These results suggest that the benefits of self-explanation over traditional practice may not be truly domain independent.

== Achievements ==
===I. Findings===
We have conducted four in-vivo studies with plans to do a fifth study during Fall 2010. The first study compared two different forms of self-explanation to a ecologically valid control condition of practice-only. We looked at both menu-based self-explanations (students chose the rule from a provided menu) and free-form responses (students typed their explanation into an empty text box). The second study compared self-explanation only to practice only in order to understand the effects that self-explanation alone had on student learning. Studies 3 and 4 introduced new instructional manipulations with goals to both increase learning and efficiency. Study 3 used analogical comparisons in an attempt to reduce the metalinguistic demands placed on the students, and Study 4 used example study in an attempt to reduce the amount of time students spent working with the tutor. The main finding from this series of studies is that while prompting students to self-explain does lead to significant learning gains, students who self-explain perform no better than students in a traditional practice condition. Furthermore, self-explanation requires significantly more time meaning it is inefficient compared to practice. These results differ from those found in previous self-explanation studies conducted largely in math and science domains.

{| class="wikitable"
|-
| [[Image:IWT_study1_learninggains.jpg]]
| [[Image:IWT_study2_learninggains.jpg]]
|-
| Fig. 1. Study 1 Learning Gains
| Fig. 2. Study 2 Learning Gains
|-
| [[Image:IWT_study3_learninggains.jpg]]
| [[Image:IWT_study4_learninggains.jpg]]
|-
| Fig. 3. Study 3 Learning Gains
| Fig. 4. Study 4 Learning Gains
|}
Figures 1-4: Across all four studies, students in all conditions show significant, but equal, learning gains. Students improve when prompted to self-explanation either alone (Study 2) or paired with tutored practice (Studies 1, 3 and 4), but there is no difference in learning between students who are prompted to self-explain and those that are not.

In addition to learning gain analysis, we have also compared efficiency scores across conditions for Studies 2, 3, and 4. Efficiency scores combine learning gains and time to complete instruction into a single measure. These results reveal a similar pattern: practice is much more efficient than self-explanation for learning the English article system.

{| class="wikitable"
|-
| [[Image:IWT_study2_efficiency.jpg]]
| [[Image:IWT_study3_efficiency.jpg]]
|-
| Fig. 5. Study 2 Efficiency Analysis
| Fig. 6. Study 3 Efficiency Analysis
|-
| [[Image:IWT_study4_efficiency.jpg]]
|-
| Fig. 7. Study 4 Learning Gains
|}
Figures 5-7: Efficiency Score Analysis. Across all studies, students in the practice-only condition were much more efficient than those in the self-explanation condition.

These studies address a key PSLC goal of identifying under what conditions instructional strategies lead to robust learning. Our results to date suggest that self-explanation may not be the best form of instruction for all domains. In addition to planning our final study, our current work investigates the effects of self-explanation prompts on robust learning measures like transfer and long-term retention.

===II. Studies===

Study 1: Pilot Study with free-form versus menu-based self-explanations
* Date: Fall 2008
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 65 students
* Total Participant Hours for the study: ~50-65 hours
* Data in the Data Shop: Yes
** [https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=241 Dataset: IWT Self-Explanation Study 0 (pilot) (Fall 2008) (raw data)]
** [https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=307 Dataset: IWT Self-Explanation Study 0 (pilot) (Fall 2008) (tutors only)]
** '''Pre/Post Test Score Data:''' No
** '''Paper or Online Tests:''' No
** '''Scanned Paper Tests:''' No
** '''Blank Tests:''' No
** '''Answer Keys: ''' No

Study 2: Self-Explanation Only versus Practice Only
* Date: Spring 2009
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 118 students
* Total Participant Hours for the study: ~100-120 hours
* Data in the Data Shop: Yes

Study 3: Analogy, Self-Explanation, and Practice
* Date: Fall 2009
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 99 students
* Total Participant Hours for the study: ~75-100 hours
* Data in the Data Shop: Yes

Study 4: Example Study, Self-Explanation, and Practice
* Date: Spring 2010
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 93 students
* Total Participant Hours for the study: ~80-95 hours
* Data in the Data Shop: Yes

Study 5: Adaptive Self-Explanation vs. Practice-Only
* Date: Fall 2010
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 142 students
* Total Participant Hours for the study: ~100 hours
* Data in the Data Shop: Yes

===Publications===

Wylie, R., Koedinger, K., and Mitamura, T. (2010) Extending the Self-Explanation Effect to Second Language Grammar Learning. International Conference of the Learning Sciences. Chicago, Illinois. June 29-July 2, 2010.

Wylie, R., Koedinger, K., and Mitamura, T. (2010) Analogies, Explanation, and Practice: Examining how task types affect second language grammar learning. Tenth International Conference on Intelligent Tutoring Systems. Pittsburgh, Pennsylvania. June 14-18, 2010.

Wylie, R., Koedinger, K., and Mitamura, T. (2009) Is Self-Explanation Always Better? The Effects of Adding Self-Explanation Prompts to an English Grammar Tutor. Cognitive Science. Amsterdam, The Netherlands. July 29 – August 1, 2009.

===Future Plans===

* Data Analysis for Study 5
* Cross-study data analysis
* Robust Learning Analysis
** Learning gains by Knowledge Component
** Long-term Retention Measure
** Transfer Measure (student writing samples)
* Paper on Efficiency Scores

[[Category:Data available in DataShop]]

Wylie - Intelligent Writing Tutor

2011-01-16T04:44:18Z

Ruth-Wylie: /* II. Studies */

== '''Project Title: Generalizing Self-Explanation''' ==
Report prepared for PSLC Site Visit, January 2011 
PSLC Research Thrust(s): Cognitive Factors 
PSLC Learnlab Course(s): English as a Second Language

{| class="wikitable" border=1
|-
| width="150pt" | PI: Ruth Wylie Department: HCII Organization: CMU
| width="150pt" | PI: Ken Koedinger Department: HCII Organization: CMU
| width="150pt" | PI: Teruko Mitamura Department: HCII Organization: CMU
|}

Others who have contributed 160 hours or more: 
PSLC Undergraduate Intern: Melissa Sheng, Rice University

{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''Study Name/Number''' || Study 1: Modalities of Self-Explanation || Study 2: Practice vs. Self-Explanation Only || Study 3: Analogy, Explanation, and Practice || Study 4: Worked Examples, Explanation, and Practice
|-
| '''Study Date''' || Fall 2008 || Spring 2009 || Fall 2009 || Spring 2010
|-
| '''LearnLab Site''' || University of Pittsburgh, ELI || University of Pittsburgh, ELI || University of Pittsburgh, ELI || University of Pittsburgh, ELI
|-
| '''LearnLab Course''' || ESL || ESL || ESL || ESL
|-
| '''Number of Students''' || 63 || 118 || 99 || 97
|-
|-
| '''DataShop''' ||
* '''Pre/Post Test Score Data:''' Yes
* '''Paper or Online Tests:''' Online
* '''Scanned Paper Tests:''' n/a
* '''Blank Tests:''' uploaded soon
* '''Answer Keys: ''' uploaded soon
||
* '''Pre/Post Test Score Data:''' Yes
* '''Paper or Online Tests:''' Online
* '''Scanned Paper Tests:''' n/a
* '''Blank Tests:''' uploaded soon
* '''Answer Keys: ''' uploaded soon
||
* '''Pre/Post Test Score Data:''' Yes
* '''Paper or Online Tests:''' Online
* '''Scanned Paper Tests:''' n/a
* '''Blank Tests:''' uploaded soon
* '''Answer Keys: ''' uploaded soon
||
* '''Pre/Post Test Score Data:''' Yes
* '''Paper or Online Tests:''' Online
* '''Scanned Paper Tests:''' n/a
* '''Blank Tests:''' uploaded soon
* '''Answer Keys: ''' uploaded soon
|}

== Project Plan Abstract ==
Prompting students to self-explain during problem solving has proven to be an effective instructional strategy across many domains. However, despite being called a “domain general” strategy, very little work has been done in areas outside of math and science. Thus, it remains an open question whether the self-explanation effect will hold in new and different domains like second language grammar learning. Through a series of classroom studies, we compare the effects of four tutoring systems: two that facilitate deep processing (prompted self-explanation and analogical comparison) and two that increase the rate of processing (example study and practice). Results show that while deep processing in general, and self-explanation specifically, is an effective strategy, it may not be an efficient approach compared to practice alone. These results suggest that the benefits of self-explanation over traditional practice may not be truly domain independent.

== Achievements ==
===I. Findings===
We have conducted four in-vivo studies with plans to do a fifth study during Fall 2010. The first study compared two different forms of self-explanation to a ecologically valid control condition of practice-only. We looked at both menu-based self-explanations (students chose the rule from a provided menu) and free-form responses (students typed their explanation into an empty text box). The second study compared self-explanation only to practice only in order to understand the effects that self-explanation alone had on student learning. Studies 3 and 4 introduced new instructional manipulations with goals to both increase learning and efficiency. Study 3 used analogical comparisons in an attempt to reduce the metalinguistic demands placed on the students, and Study 4 used example study in an attempt to reduce the amount of time students spent working with the tutor. The main finding from this series of studies is that while prompting students to self-explain does lead to significant learning gains, students who self-explain perform no better than students in a traditional practice condition. Furthermore, self-explanation requires significantly more time meaning it is inefficient compared to practice. These results differ from those found in previous self-explanation studies conducted largely in math and science domains.

{| class="wikitable"
|-
| [[Image:IWT_study1_learninggains.jpg]]
| [[Image:IWT_study2_learninggains.jpg]]
|-
| Fig. 1. Study 1 Learning Gains
| Fig. 2. Study 2 Learning Gains
|-
| [[Image:IWT_study3_learninggains.jpg]]
| [[Image:IWT_study4_learninggains.jpg]]
|-
| Fig. 3. Study 3 Learning Gains
| Fig. 4. Study 4 Learning Gains
|}
Figures 1-4: Across all four studies, students in all conditions show significant, but equal, learning gains. Students improve when prompted to self-explanation either alone (Study 2) or paired with tutored practice (Studies 1, 3 and 4), but there is no difference in learning between students who are prompted to self-explain and those that are not.

In addition to learning gain analysis, we have also compared efficiency scores across conditions for Studies 2, 3, and 4. Efficiency scores combine learning gains and time to complete instruction into a single measure. These results reveal a similar pattern: practice is much more efficient than self-explanation for learning the English article system.

{| class="wikitable"
|-
| [[Image:IWT_study2_efficiency.jpg]]
| [[Image:IWT_study3_efficiency.jpg]]
|-
| Fig. 5. Study 2 Efficiency Analysis
| Fig. 6. Study 3 Efficiency Analysis
|-
| [[Image:IWT_study4_efficiency.jpg]]
|-
| Fig. 7. Study 4 Learning Gains
|}
Figures 5-7: Efficiency Score Analysis. Across all studies, students in the practice-only condition were much more efficient than those in the self-explanation condition.

These studies address a key PSLC goal of identifying under what conditions instructional strategies lead to robust learning. Our results to date suggest that self-explanation may not be the best form of instruction for all domains. In addition to planning our final study, our current work investigates the effects of self-explanation prompts on robust learning measures like transfer and long-term retention.

===II. Studies===

Study 1: Pilot Study with free-form versus menu-based self-explanations
* Date: Fall 2008
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 65 students
* Total Participant Hours for the study: ~50-65 hours
* Data in the Data Shop: Yes
** [https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=241 Dataset: IWT Self-Explanation Study 0 (pilot) (Fall 2008) (raw data)]
** [https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=307 Dataset: IWT Self-Explanation Study 0 (pilot) (Fall 2008) (tutors only)]
** '''Pre/Post Test Score Data:''' No
** '''Paper or Online Tests:''' No
** '''Scanned Paper Tests:''' No
** '''Blank Tests:''' No
** '''Answer Keys: ''' No

Study 2: Self-Explanation Only versus Practice Only
* Date: Spring 2009
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 118 students
* Total Participant Hours for the study: ~100-120 hours
* Data in the Data Shop: Yes

Study 3: Analogy, Self-Explanation, and Practice
* Date: Fall 2009
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 99 students
* Total Participant Hours for the study: ~75-100 hours
* Data in the Data Shop: Yes

Study 4: Example Study, Self-Explanation, and Practice
* Date: Spring 2010
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 93 students
* Total Participant Hours for the study: ~80-95 hours
* Data in the Data Shop: Yes

Study 5: Adaptive Self-Explanation vs. Practice-Only
* Date: Fall 2010
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 142 students
* Total Participant Hours for the study: ~100 hours
* Data in the Data Shop: Yes

===Publications===

Wylie, R., Koedinger, K., and Mitamura, T. (2010) Extending the Self-Explanation Effect to Second Language Grammar Learning. International Conference of the Learning Sciences. Chicago, Illinois. June 29-July 2, 2010.

Wylie, R., Koedinger, K., and Mitamura, T. (2010) Analogies, Explanation, and Practice: Examining how task types affect second language grammar learning. Tenth International Conference on Intelligent Tutoring Systems. Pittsburgh, Pennsylvania. June 14-18, 2010.

Wylie, R., Koedinger, K., and Mitamura, T. (2009) Is Self-Explanation Always Better? The Effects of Adding Self-Explanation Prompts to an English Grammar Tutor. Cognitive Science. Amsterdam, The Netherlands. July 29 – August 1, 2009.

===Future Plans===

* Data Analysis for Study 5
* Cross-study data analysis
* Robust Learning Analysis
** Learning gains by Knowledge Component
** Long-term Retention Measure
** Transfer Measure (student writing samples)
* Paper on Efficiency Scores

[[Category:Data available in DataShop]]

CF December 2010

2010-12-08T19:04:41Z

Ruth-Wylie:

No major business to discuss, but make sure the following dates are on your calendar:

AB Rehearsal: January 14 
AB Visit: January 20-21 
iSLC: March 25-27
 

Talk by Oscar Saz gave a talk:
"Language learning in a special education environment".

Abstract: This talk addresses the development of a computer-based tool for providing language learning to young Spanish students with difficulties in their L1, focusing on the situation of students with development disorders attending special education. The requirements of a tool of this kind are many, as it has to be especially motivational for the students while providing supportive feedback according to their cognitive capabilities. How these requirements were addressed and the results of the use of the developed tool are shown. Furthermore, a study on the production of lexical errors in the speech of students with development delays is reported, showing how their lexical capabilities are comparable to those of healthy unimpaired children in the 3-5 year old range.

Slides here: [[Media:CF_Saz_Dec8.pdf‎]]

Next Meeting: probably February, post AB visit but all are welcome to the Cognitive Factors AB rehearsal

CF December 2010

2010-12-08T18:57:37Z

Ruth-Wylie:

CF December 2010

2010-12-08T18:56:30Z

Ruth-Wylie:

No major business to discuss, but make sure the following dates are on your calendar:

AB Rehearsal: January 14
AB Visit: January 20-21

iSLC: March 25-27

Talk by Oscar Saz gave a talk:
"Language learning in a special education environment".

Abstract: This talk addresses the development of a computer-based tool for providing language learning to young Spanish students with difficulties in their L1, focusing on the situation of students with development disorders attending special education. The requirements of a tool of this kind are many, as it has to be especially motivational for the students while providing supportive feedback according to their cognitive capabilities. How these requirements were addressed and the results of the use of the developed tool are shown. Furthermore, a study on the production of lexical errors in the speech of students with development delays is reported, showing how their lexical capabilities are comparable to those of healthy unimpaired children in the 3-5 year old range.

Slides here: [[Media:CF_Saz_Dec8.pdf‎]]

File:CF Saz Dec8.pdf

2010-12-08T18:55:21Z

Ruth-Wylie:

CF December 2010

2010-12-08T18:54:20Z

Ruth-Wylie: New page: No major business to discuss, but make sure the following dates are on your calendar: AB Rehearsal: January 14 AB Visit: January 20-21 iSLC: March 25-27 Talk by Oscar Saz gave a talk: "...

Cognitive Factors Monthly Meeting Notes

2010-12-08T18:50:28Z

Ruth-Wylie:

Notes from the monthly meetings will be posted here, along with copies of slides and handouts. Feel free to add and correct as you see fit.

[[CF September 2010]] September 24, 2010, Pitt 
[[CF November 2010]] November, 16, 2010, CMU 
[[CF December 2010]] December 8, 2010, Pitt

CF November 2010

2010-11-19T20:11:32Z

Ruth-Wylie:

Meeting Notes – 9.16.10

Posters for AB Visit (Mary Lou collected volunteers, send her an email ASAP if you want to present your work)

AB Visit – January 20-21, 2010

Talk by Colleen Davy: Using an Oral Repetition Task to Develop Fluent Second Language Speaking Skills 
Slides here: [[Media:CF OralProduction Nov16.pdf]]

Next meeting: Wednesday, December 8. 9-10:30am, 9th Floor LRDC
Oscar Saz will be giving a talk.

Also, check out Nora's proposal talk: Friday, December 3, 4:00

CF November 2010

2010-11-19T20:10:54Z

Ruth-Wylie:

Meeting Notes – 9.16.10

Posters for AB Visit (Mary Lou collected volunteers, send her an email ASAP if you want to present your work)

AB Visit – January 20-21, 2010

Talk by Colleen Davy: Using an Oral Repetition Task to Develop Fluent Second Language Speaking Skills 
Slides here: [[Image:CF OralProduction Nov16.pdf]]

Next meeting: Wednesday, December 8. 9-10:30am, 9th Floor LRDC
Oscar Saz will be giving a talk.

Also, check out Nora's proposal talk: Friday, December 3, 4:00

CF November 2010

2010-11-19T20:09:31Z

Ruth-Wylie:

Meeting Notes – 9.16.10

Posters for AB Visit (Mary Lou collected volunteers, send her an email ASAP if you want to present your work)

AB Visit – January 20-21, 2010

Talk by Colleen Davy: Using an Oral Repetition Task to Develop Fluent Second Language Speaking Skills 
Slides here: [[CF_OralProduction_Nov16.pdf]]

Next meeting: Wednesday, December 8. 9-10:30am, 9th Floor LRDC
Oscar Saz will be giving a talk.

Also, check out Nora's proposal talk: Friday, December 3, 4:00

File:CF OralProduction Nov16.pdf

2010-11-19T20:08:33Z

Ruth-Wylie: Colleen Davy's slides from CF November 2010 meeting.

Colleen Davy's slides from CF November 2010 meeting.

CF November 2010

2010-11-19T19:34:03Z

Ruth-Wylie:

Meeting Notes – 9.16.10

Posters for AB Visit (Mary Lou collected volunteers, send her an email ASAP if you want to present your work)

AB Visit – January 20-21, 2010

Talk by Colleen Davy: Using an Oral Repetition Task to Develop Fluent Second Language Speaking Skills

Next meeting: Wednesday, December 8. 9-10:30am, 9th Floor LRDC
Oscar Saz will be giving a talk.

Also, check out Nora's proposal talk: Friday, December 3, 4:00

CF November 2010

2010-11-16T16:17:14Z

Ruth-Wylie: New page: Meeting Notes – 9.16.10 Posters for AB Visit (Mary Lou collected volunteers, send her an email ASAP if you want to present your work) AB Visit – January 20-21, 2010 Talk by Colleen ...

Meeting Notes – 9.16.10

Posters for AB Visit (Mary Lou collected volunteers, send her an email ASAP if you want to present your work)

AB Visit – January 20-21, 2010

Talk by Colleen Davy: Using an Oral Repetition Task to Develop Fluent Second Language Speaking Skills

No meeting in December. Check out Nora's proposal talk instead: Friday, December 3, 4:00

Cognitive Factors Monthly Meeting Notes

2010-11-16T16:14:18Z

Ruth-Wylie:

Cognitive Factors Monthly Meeting Notes

2010-11-16T16:13:57Z

Ruth-Wylie:

Wylie - Intelligent Writing Tutor

2010-11-09T14:48:28Z

Ruth-Wylie: /* Plans for January 2011 - June 2010 */

== '''Project Title: Generalizing Self-Explanation''' ==
Report prepared for PSLC Site Visit, January 2011 
PSLC Research Thrust(s): Cognitive Factors 
PSLC Learnlab Course(s): English as a Second Language

{| class="wikitable" border=1
|-
| width="150pt" | PI: Ruth Wylie Department: HCII Organization: CMU
| width="150pt" | PI: Ken Koedinger Department: HCII Organization: CMU
| width="150pt" | PI: Teruko Mitamura Department: HCII Organization: CMU
|}

Others who have contributed 160 hours or more: 
PSLC Undergraduate Intern: Melissa Sheng, Rice University

{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''Study Name/Number''' || Study 1: Modalities of Self-Explanation || Study 2: Practice vs. Self-Explanation Only || Study 3: Analogy, Explanation, and Practice || Study 4: Worked Examples, Explanation, and Practice
|-
| '''Study Date''' || Fall 2008 || Spring 2009 || Fall 2009 || Spring 2010
|-
| '''LearnLab Site''' || University of Pittsburgh, ELI || University of Pittsburgh, ELI || University of Pittsburgh, ELI || University of Pittsburgh, ELI
|-
| '''LearnLab Course''' || ESL || ESL || ESL || ESL
|-
| '''Number of Students''' || 63 || 118 || 99 || 97
|-
|-
| '''DataShop''' ||
* '''Pre/Post Test Score Data:''' Yes
* '''Paper or Online Tests:''' Online
* '''Scanned Paper Tests:''' n/a
* '''Blank Tests:''' uploaded soon
* '''Answer Keys: ''' uploaded soon
||
* '''Pre/Post Test Score Data:''' Yes
* '''Paper or Online Tests:''' Online
* '''Scanned Paper Tests:''' n/a
* '''Blank Tests:''' uploaded soon
* '''Answer Keys: ''' uploaded soon
||
* '''Pre/Post Test Score Data:''' Yes
* '''Paper or Online Tests:''' Online
* '''Scanned Paper Tests:''' n/a
* '''Blank Tests:''' uploaded soon
* '''Answer Keys: ''' uploaded soon
||
* '''Pre/Post Test Score Data:''' Yes
* '''Paper or Online Tests:''' Online
* '''Scanned Paper Tests:''' n/a
* '''Blank Tests:''' uploaded soon
* '''Answer Keys: ''' uploaded soon
|}

== Project Plan Abstract ==
Prompting students to self-explain during problem solving has proven to be an effective instructional strategy across many domains. However, despite being called a “domain general” strategy, very little work has been done in areas outside of math and science. Thus, it remains an open question whether the self-explanation effect will hold in new and different domains like second language grammar learning. Through a series of classroom studies, we compare the effects of four tutoring systems: two that facilitate deep processing (prompted self-explanation and analogical comparison) and two that increase the rate of processing (example study and practice). Results show that while deep processing in general, and self-explanation specifically, is an effective strategy, it may not be an efficient approach compared to practice alone. These results suggest that the benefits of self-explanation over traditional practice may not be truly domain independent.

== Achievements ==
===I. Findings===
We have conducted four in-vivo studies with plans to do a fifth study during Fall 2010. The first study compared two different forms of self-explanation to a ecologically valid control condition of practice-only. We looked at both menu-based self-explanations (students chose the rule from a provided menu) and free-form responses (students typed their explanation into an empty text box). The second study compared self-explanation only to practice only in order to understand the effects that self-explanation alone had on student learning. Studies 3 and 4 introduced new instructional manipulations with goals to both increase learning and efficiency. Study 3 used analogical comparisons in an attempt to reduce the metalinguistic demands placed on the students, and Study 4 used example study in an attempt to reduce the amount of time students spent working with the tutor. The main finding from this series of studies is that while prompting students to self-explain does lead to significant learning gains, students who self-explain perform no better than students in a traditional practice condition. Furthermore, self-explanation requires significantly more time meaning it is inefficient compared to practice. These results differ from those found in previous self-explanation studies conducted largely in math and science domains.

{| class="wikitable"
|-
| [[Image:IWT_study1_learninggains.jpg]]
| [[Image:IWT_study2_learninggains.jpg]]
|-
| Fig. 1. Study 1 Learning Gains
| Fig. 2. Study 2 Learning Gains
|-
| [[Image:IWT_study3_learninggains.jpg]]
| [[Image:IWT_study4_learninggains.jpg]]
|-
| Fig. 3. Study 3 Learning Gains
| Fig. 4. Study 4 Learning Gains
|}
Figures 1-4: Across all four studies, students in all conditions show significant, but equal, learning gains. Students improve when prompted to self-explanation either alone (Study 2) or paired with tutored practice (Studies 1, 3 and 4), but there is no difference in learning between students who are prompted to self-explain and those that are not.

In addition to learning gain analysis, we have also compared efficiency scores across conditions for Studies 2, 3, and 4. Efficiency scores combine learning gains and time to complete instruction into a single measure. These results reveal a similar pattern: practice is much more efficient than self-explanation for learning the English article system.

{| class="wikitable"
|-
| [[Image:IWT_study2_efficiency.jpg]]
| [[Image:IWT_study3_efficiency.jpg]]
|-
| Fig. 5. Study 2 Efficiency Analysis
| Fig. 6. Study 3 Efficiency Analysis
|-
| [[Image:IWT_study4_efficiency.jpg]]
|-
| Fig. 7. Study 4 Learning Gains
|}
Figures 5-7: Efficiency Score Analysis. Across all studies, students in the practice-only condition were much more efficient than those in the self-explanation condition.

These studies address a key PSLC goal of identifying under what conditions instructional strategies lead to robust learning. Our results to date suggest that self-explanation may not be the best form of instruction for all domains. In addition to planning our final study, our current work investigates the effects of self-explanation prompts on robust learning measures like transfer and long-term retention.

===II. Studies===

Study 1: Pilot Study with free-form versus menu-based self-explanations
* Date: Fall 2008
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 65 students
* Total Participant Hours for the study: ~50-65 hours
* Data in the Data Shop: Yes
** [https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=241 Dataset: IWT Self-Explanation Study 0 (pilot) (Fall 2008) (raw data)]
** [https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=307 Dataset: IWT Self-Explanation Study 0 (pilot) (Fall 2008) (tutors only)]
** '''Pre/Post Test Score Data:''' No
** '''Paper or Online Tests:''' No
** '''Scanned Paper Tests:''' No
** '''Blank Tests:''' No
** '''Answer Keys: ''' No

Study 2: Self-Explanation Only versus Practice Only
* Date: Spring 2009
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 118 students
* Total Participant Hours for the study: ~100-120 hours
* Data in the Data Shop: Yes

Study 3: Analogy, Self-Explanation, and Practice
* Date: Fall 2009
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 99 students
* Total Participant Hours for the study: ~75-100 hours
* Data in the Data Shop: Yes

Study 4: Pilot Study with free-form versus menu-based self-explanations
* Date: Spring 2010
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 93 students
* Total Participant Hours for the study: ~80-95 hours
* Data in the Data Shop: Yes

Study 5: Pilot Study with free-form versus menu-based self-explanations
* Date: Planned Fall 2010
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: ~100 students
* Total Participant Hours for the study: ~100 hours
* Data in the Data Shop: Will be automatically logged by CTAT tutors

===Publications===

Wylie, R., Koedinger, K., and Mitamura, T. (2010) Extending the Self-Explanation Effect to Second Language Grammar Learning. International Conference of the Learning Sciences. Chicago, Illinois. June 29-July 2, 2010.

Wylie, R., Koedinger, K., and Mitamura, T. (2010) Analogies, Explanation, and Practice: Examining how task types affect second language grammar learning. Tenth International Conference on Intelligent Tutoring Systems. Pittsburgh, Pennsylvania. June 14-18, 2010.

Wylie, R., Koedinger, K., and Mitamura, T. (2009) Is Self-Explanation Always Better? The Effects of Adding Self-Explanation Prompts to an English Grammar Tutor. Cognitive Science. Amsterdam, The Netherlands. July 29 – August 1, 2009.

===Future Plans===

* Data Analysis for Study 5
* Cross-study data analysis
* Robust Learning Analysis
** Learning gains by Knowledge Component
** Long-term Retention Measure
** Transfer Measure (student writing samples)
* Paper on Efficiency Scores

Wylie - Intelligent Writing Tutor

2010-11-09T14:46:33Z

Ruth-Wylie: /* '''Project Title: Generalizing Self-Explanation''' */

== '''Project Title: Generalizing Self-Explanation''' ==
Report prepared for PSLC Site Visit, January 2011 
PSLC Research Thrust(s): Cognitive Factors 
PSLC Learnlab Course(s): English as a Second Language

{| class="wikitable" border=1
|-
| width="150pt" | PI: Ruth Wylie Department: HCII Organization: CMU
| width="150pt" | PI: Ken Koedinger Department: HCII Organization: CMU
| width="150pt" | PI: Teruko Mitamura Department: HCII Organization: CMU
|}

Others who have contributed 160 hours or more: 
PSLC Undergraduate Intern: Melissa Sheng, Rice University

{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''Study Name/Number''' || Study 1: Modalities of Self-Explanation || Study 2: Practice vs. Self-Explanation Only || Study 3: Analogy, Explanation, and Practice || Study 4: Worked Examples, Explanation, and Practice
|-
| '''Study Date''' || Fall 2008 || Spring 2009 || Fall 2009 || Spring 2010
|-
| '''LearnLab Site''' || University of Pittsburgh, ELI || University of Pittsburgh, ELI || University of Pittsburgh, ELI || University of Pittsburgh, ELI
|-
| '''LearnLab Course''' || ESL || ESL || ESL || ESL
|-
| '''Number of Students''' || 63 || 118 || 99 || 97
|-
|-
| '''DataShop''' ||
* '''Pre/Post Test Score Data:''' Yes
* '''Paper or Online Tests:''' Online
* '''Scanned Paper Tests:''' n/a
* '''Blank Tests:''' uploaded soon
* '''Answer Keys: ''' uploaded soon
||
* '''Pre/Post Test Score Data:''' Yes
* '''Paper or Online Tests:''' Online
* '''Scanned Paper Tests:''' n/a
* '''Blank Tests:''' uploaded soon
* '''Answer Keys: ''' uploaded soon
||
* '''Pre/Post Test Score Data:''' Yes
* '''Paper or Online Tests:''' Online
* '''Scanned Paper Tests:''' n/a
* '''Blank Tests:''' uploaded soon
* '''Answer Keys: ''' uploaded soon
||
* '''Pre/Post Test Score Data:''' Yes
* '''Paper or Online Tests:''' Online
* '''Scanned Paper Tests:''' n/a
* '''Blank Tests:''' uploaded soon
* '''Answer Keys: ''' uploaded soon
|}

== Project Plan Abstract ==
Prompting students to self-explain during problem solving has proven to be an effective instructional strategy across many domains. However, despite being called a “domain general” strategy, very little work has been done in areas outside of math and science. Thus, it remains an open question whether the self-explanation effect will hold in new and different domains like second language grammar learning. Through a series of classroom studies, we compare the effects of four tutoring systems: two that facilitate deep processing (prompted self-explanation and analogical comparison) and two that increase the rate of processing (example study and practice). Results show that while deep processing in general, and self-explanation specifically, is an effective strategy, it may not be an efficient approach compared to practice alone. These results suggest that the benefits of self-explanation over traditional practice may not be truly domain independent.

== Achievements ==
===I. Findings===
We have conducted four in-vivo studies with plans to do a fifth study during Fall 2010. The first study compared two different forms of self-explanation to a ecologically valid control condition of practice-only. We looked at both menu-based self-explanations (students chose the rule from a provided menu) and free-form responses (students typed their explanation into an empty text box). The second study compared self-explanation only to practice only in order to understand the effects that self-explanation alone had on student learning. Studies 3 and 4 introduced new instructional manipulations with goals to both increase learning and efficiency. Study 3 used analogical comparisons in an attempt to reduce the metalinguistic demands placed on the students, and Study 4 used example study in an attempt to reduce the amount of time students spent working with the tutor. The main finding from this series of studies is that while prompting students to self-explain does lead to significant learning gains, students who self-explain perform no better than students in a traditional practice condition. Furthermore, self-explanation requires significantly more time meaning it is inefficient compared to practice. These results differ from those found in previous self-explanation studies conducted largely in math and science domains.

{| class="wikitable"
|-
| [[Image:IWT_study1_learninggains.jpg]]
| [[Image:IWT_study2_learninggains.jpg]]
|-
| Fig. 1. Study 1 Learning Gains
| Fig. 2. Study 2 Learning Gains
|-
| [[Image:IWT_study3_learninggains.jpg]]
| [[Image:IWT_study4_learninggains.jpg]]
|-
| Fig. 3. Study 3 Learning Gains
| Fig. 4. Study 4 Learning Gains
|}
Figures 1-4: Across all four studies, students in all conditions show significant, but equal, learning gains. Students improve when prompted to self-explanation either alone (Study 2) or paired with tutored practice (Studies 1, 3 and 4), but there is no difference in learning between students who are prompted to self-explain and those that are not.

In addition to learning gain analysis, we have also compared efficiency scores across conditions for Studies 2, 3, and 4. Efficiency scores combine learning gains and time to complete instruction into a single measure. These results reveal a similar pattern: practice is much more efficient than self-explanation for learning the English article system.

{| class="wikitable"
|-
| [[Image:IWT_study2_efficiency.jpg]]
| [[Image:IWT_study3_efficiency.jpg]]
|-
| Fig. 5. Study 2 Efficiency Analysis
| Fig. 6. Study 3 Efficiency Analysis
|-
| [[Image:IWT_study4_efficiency.jpg]]
|-
| Fig. 7. Study 4 Learning Gains
|}
Figures 5-7: Efficiency Score Analysis. Across all studies, students in the practice-only condition were much more efficient than those in the self-explanation condition.

These studies address a key PSLC goal of identifying under what conditions instructional strategies lead to robust learning. Our results to date suggest that self-explanation may not be the best form of instruction for all domains. In addition to planning our final study, our current work investigates the effects of self-explanation prompts on robust learning measures like transfer and long-term retention.

===II. Studies===

Study 1: Pilot Study with free-form versus menu-based self-explanations
* Date: Fall 2008
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 65 students
* Total Participant Hours for the study: ~50-65 hours
* Data in the Data Shop: Yes
** [https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=241 Dataset: IWT Self-Explanation Study 0 (pilot) (Fall 2008) (raw data)]
** [https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=307 Dataset: IWT Self-Explanation Study 0 (pilot) (Fall 2008) (tutors only)]
** '''Pre/Post Test Score Data:''' No
** '''Paper or Online Tests:''' No
** '''Scanned Paper Tests:''' No
** '''Blank Tests:''' No
** '''Answer Keys: ''' No

Study 2: Self-Explanation Only versus Practice Only
* Date: Spring 2009
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 118 students
* Total Participant Hours for the study: ~100-120 hours
* Data in the Data Shop: Yes

Study 3: Analogy, Self-Explanation, and Practice
* Date: Fall 2009
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 99 students
* Total Participant Hours for the study: ~75-100 hours
* Data in the Data Shop: Yes

Study 4: Pilot Study with free-form versus menu-based self-explanations
* Date: Spring 2010
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 93 students
* Total Participant Hours for the study: ~80-95 hours
* Data in the Data Shop: Yes

Study 5: Pilot Study with free-form versus menu-based self-explanations
* Date: Planned Fall 2010
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: ~100 students
* Total Participant Hours for the study: ~100 hours
* Data in the Data Shop: Will be automatically logged by CTAT tutors

===Publications===

Wylie, R., Koedinger, K., and Mitamura, T. (2010) Extending the Self-Explanation Effect to Second Language Grammar Learning. International Conference of the Learning Sciences. Chicago, Illinois. June 29-July 2, 2010.

Wylie, R., Koedinger, K., and Mitamura, T. (2010) Analogies, Explanation, and Practice: Examining how task types affect second language grammar learning. Tenth International Conference on Intelligent Tutoring Systems. Pittsburgh, Pennsylvania. June 14-18, 2010.

Wylie, R., Koedinger, K., and Mitamura, T. (2009) Is Self-Explanation Always Better? The Effects of Adding Self-Explanation Prompts to an English Grammar Tutor. Cognitive Science. Amsterdam, The Netherlands. July 29 – August 1, 2009.

===Plans for January 2011 - June 2010===

* Data Analysis for Study 5
* Cross-study data analysis
* Robust Learning Analysis
** Learning gains by Knowledge Component
** Long-term Retention Measure
** Transfer Measure (student writing samples)
* Paper on Efficiency Scores

Wylie - Intelligent Writing Tutor

2010-11-09T14:42:23Z

Ruth-Wylie: /* '''Project Title: Generalizing Self-Explanation''' */

== '''Project Title: Generalizing Self-Explanation''' ==
Report prepared for PSLC Site Visit, January 2011 
PSLC Research Thrust(s): Cognitive Factors 
PSLC Learnlab Course(s): English as a Second Language

{| class="wikitable" border=1
|-
| width="150pt" | PI: Ruth Wylie Department: HCII Organization: CMU
| width="150pt" | PI: Ken Koedinger Department: HCII Organization: CMU
| width="150pt" | PI: Teruko Mitamura Department: HCII Organization: CMU
|}

Others who have contributed 160 hours or more: 
PSLC Undergraduate Intern: Melissa Sheng, Rice University

{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
| '''Study Name/Number''' || Pilot Study (In Vivo) || Study 1: Practice vs. Self-Explanation Only || Study 2a: Analogy, Explanation, and Practice || Study 2b: Worked Examples, Explanation, and Practice
|-
| '''Study Date''' || Fall 2008 || Spring 2009 || Fall 2009 || Spring 2010
|-
| '''LearnLab Site''' || University of Pittsburgh, ELI || University of Pittsburgh, ELI || University of Pittsburgh, ELI || University of Pittsburgh, ELI
|-
| '''LearnLab Course''' || ESL || ESL || ESL || ESL
|-
| '''Number of Students''' || 63 || 118 || 99 || 97
|-
|-
| '''DataShop''' ||
* '''Pre/Post Test Score Data:''' Yes
* '''Paper or Online Tests:''' Online
* '''Scanned Paper Tests:''' n/a
* '''Blank Tests:''' uploaded soon
* '''Answer Keys: ''' uploaded soon
||
* '''Pre/Post Test Score Data:''' Yes
* '''Paper or Online Tests:''' Online
* '''Scanned Paper Tests:''' n/a
* '''Blank Tests:''' uploaded soon
* '''Answer Keys: ''' uploaded soon
||
* '''Pre/Post Test Score Data:''' Yes
* '''Paper or Online Tests:''' Online
* '''Scanned Paper Tests:''' n/a
* '''Blank Tests:''' uploaded soon
* '''Answer Keys: ''' uploaded soon
||
* '''Pre/Post Test Score Data:''' Yes
* '''Paper or Online Tests:''' Online
* '''Scanned Paper Tests:''' n/a
* '''Blank Tests:''' uploaded soon
* '''Answer Keys: ''' uploaded soon
|}

== Project Plan Abstract ==
Prompting students to self-explain during problem solving has proven to be an effective instructional strategy across many domains. However, despite being called a “domain general” strategy, very little work has been done in areas outside of math and science. Thus, it remains an open question whether the self-explanation effect will hold in new and different domains like second language grammar learning. Through a series of classroom studies, we compare the effects of four tutoring systems: two that facilitate deep processing (prompted self-explanation and analogical comparison) and two that increase the rate of processing (example study and practice). Results show that while deep processing in general, and self-explanation specifically, is an effective strategy, it may not be an efficient approach compared to practice alone. These results suggest that the benefits of self-explanation over traditional practice may not be truly domain independent.

== Achievements ==
===I. Findings===
We have conducted four in-vivo studies with plans to do a fifth study during Fall 2010. The first study compared two different forms of self-explanation to a ecologically valid control condition of practice-only. We looked at both menu-based self-explanations (students chose the rule from a provided menu) and free-form responses (students typed their explanation into an empty text box). The second study compared self-explanation only to practice only in order to understand the effects that self-explanation alone had on student learning. Studies 3 and 4 introduced new instructional manipulations with goals to both increase learning and efficiency. Study 3 used analogical comparisons in an attempt to reduce the metalinguistic demands placed on the students, and Study 4 used example study in an attempt to reduce the amount of time students spent working with the tutor. The main finding from this series of studies is that while prompting students to self-explain does lead to significant learning gains, students who self-explain perform no better than students in a traditional practice condition. Furthermore, self-explanation requires significantly more time meaning it is inefficient compared to practice. These results differ from those found in previous self-explanation studies conducted largely in math and science domains.

{| class="wikitable"
|-
| [[Image:IWT_study1_learninggains.jpg]]
| [[Image:IWT_study2_learninggains.jpg]]
|-
| Fig. 1. Study 1 Learning Gains
| Fig. 2. Study 2 Learning Gains
|-
| [[Image:IWT_study3_learninggains.jpg]]
| [[Image:IWT_study4_learninggains.jpg]]
|-
| Fig. 3. Study 3 Learning Gains
| Fig. 4. Study 4 Learning Gains
|}
Figures 1-4: Across all four studies, students in all conditions show significant, but equal, learning gains. Students improve when prompted to self-explanation either alone (Study 2) or paired with tutored practice (Studies 1, 3 and 4), but there is no difference in learning between students who are prompted to self-explain and those that are not.

In addition to learning gain analysis, we have also compared efficiency scores across conditions for Studies 2, 3, and 4. Efficiency scores combine learning gains and time to complete instruction into a single measure. These results reveal a similar pattern: practice is much more efficient than self-explanation for learning the English article system.

{| class="wikitable"
|-
| [[Image:IWT_study2_efficiency.jpg]]
| [[Image:IWT_study3_efficiency.jpg]]
|-
| Fig. 5. Study 2 Efficiency Analysis
| Fig. 6. Study 3 Efficiency Analysis
|-
| [[Image:IWT_study4_efficiency.jpg]]
|-
| Fig. 7. Study 4 Learning Gains
|}
Figures 5-7: Efficiency Score Analysis. Across all studies, students in the practice-only condition were much more efficient than those in the self-explanation condition.

These studies address a key PSLC goal of identifying under what conditions instructional strategies lead to robust learning. Our results to date suggest that self-explanation may not be the best form of instruction for all domains. In addition to planning our final study, our current work investigates the effects of self-explanation prompts on robust learning measures like transfer and long-term retention.

===II. Studies===

Study 1: Pilot Study with free-form versus menu-based self-explanations
* Date: Fall 2008
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 65 students
* Total Participant Hours for the study: ~50-65 hours
* Data in the Data Shop: Yes
** [https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=241 Dataset: IWT Self-Explanation Study 0 (pilot) (Fall 2008) (raw data)]
** [https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=307 Dataset: IWT Self-Explanation Study 0 (pilot) (Fall 2008) (tutors only)]
** '''Pre/Post Test Score Data:''' No
** '''Paper or Online Tests:''' No
** '''Scanned Paper Tests:''' No
** '''Blank Tests:''' No
** '''Answer Keys: ''' No

Study 2: Self-Explanation Only versus Practice Only
* Date: Spring 2009
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 118 students
* Total Participant Hours for the study: ~100-120 hours
* Data in the Data Shop: Yes

Study 3: Analogy, Self-Explanation, and Practice
* Date: Fall 2009
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 99 students
* Total Participant Hours for the study: ~75-100 hours
* Data in the Data Shop: Yes

Study 4: Pilot Study with free-form versus menu-based self-explanations
* Date: Spring 2010
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: 93 students
* Total Participant Hours for the study: ~80-95 hours
* Data in the Data Shop: Yes

Study 5: Pilot Study with free-form versus menu-based self-explanations
* Date: Planned Fall 2010
* LearnLab Site and Courses: English LearnLab, Grammar
* Number of Students: ~100 students
* Total Participant Hours for the study: ~100 hours
* Data in the Data Shop: Will be automatically logged by CTAT tutors

===Publications===

Wylie, R., Koedinger, K., and Mitamura, T. (2010) Extending the Self-Explanation Effect to Second Language Grammar Learning. International Conference of the Learning Sciences. Chicago, Illinois. June 29-July 2, 2010.

Wylie, R., Koedinger, K., and Mitamura, T. (2010) Analogies, Explanation, and Practice: Examining how task types affect second language grammar learning. Tenth International Conference on Intelligent Tutoring Systems. Pittsburgh, Pennsylvania. June 14-18, 2010.

Wylie, R., Koedinger, K., and Mitamura, T. (2009) Is Self-Explanation Always Better? The Effects of Adding Self-Explanation Prompts to an English Grammar Tutor. Cognitive Science. Amsterdam, The Netherlands. July 29 – August 1, 2009.

===Plans for January 2011 - June 2010===

* Data Analysis for Study 5
* Cross-study data analysis
* Robust Learning Analysis
** Learning gains by Knowledge Component
** Long-term Retention Measure
** Transfer Measure (student writing samples)
* Paper on Efficiency Scores

CF September 2010

2010-09-24T22:18:03Z

Ruth-Wylie:

September 24, 2010

Welcome to the new members!

Send Ruth email if you (or any new collaborators, post-docs, grad students) need to be added to the cognitive factors d-list

Send Jo email if you need to be added to the general PSLC d-list

Advisory board dates - January 20 & 21, 2011 (Thur and Fri)

Speaker Series
- Rob Goldstone has agreed to come (probably before the AB)

Handout: Cognitive Factors Thrust Plan, if see you see errors send them to Chuck (link to document coming soon)

In general for Annual Report and Strategic Plan it is important to have non-text contributions; send screenshots/pictures of interventions and/or graphs of results as they come up

Also as a general reminder, it is never too early to send bullets of exciting findings, usually collected at least once a year

Talk: How does learning to write help learning to read Chinese (fMRI study) - Fan Cao
Abstract
Two types of instructions were given to a group of English speakers who learn Chinese as a L2. One is character writing and the other is pinyin writing. The hypothesis is that writing will facilitate the integration of orthographic, phonological and semantic representations by involving both perception and production and by emphasizing the special features of Chinese
characters. fMRI scans found that sensory-motor cortex and visual-spatial representation cortex are more involved if the subject had writing experience. We also found that writing training produced more elaborated representations of orthography, phonology and semantics in the brain as compared to pinyin training.

Slides here: [[Media:PSLC_Sep_24_1.pdf]]

Next up: Colleen Davy will speak at the October meeting, likely the last week of Oct at CMU

File:PSLC Sep 24 1.pdf

2010-09-24T22:12:49Z

Ruth-Wylie: Slide from Fan's talk (09.24.10)

Slide from Fan's talk (09.24.10)

Cognitive Factors Monthly Meeting Notes

2010-09-24T21:11:21Z

Ruth-Wylie:

Notes from the monthly meetings will be posted here, along with copies of slides and handouts. Feel free to add and correct as you see fit.

[[CF September 2010]] September 24, 2010, LRDC

CF September 2010

2010-09-24T21:10:31Z

Ruth-Wylie: New page: September 24, 2010 Welcome to the new members! Send Ruth email if you (or any new collaborators, post-docs, grad students) need to be added to the cognitive factors d-list Send Jo email...

September 24, 2010

Welcome to the new members!

Send Ruth email if you (or any new collaborators, post-docs, grad students) need to be added to the cognitive factors d-list

Send Jo email if you need to be added to the general PSLC d-list

Advisory board dates - January 20 & 21, 2011 (Thur and Fri)

Speaker Series
- Rob Goldstone has agreed to come (probably before the AB)

Handout: Cognitive Factors Thrust Plan, if see you see errors send them to Chuck (link to document coming soon)

In general for Annual Report and Strategic Plan it is important to have non-text contributions; send screenshots/pictures of interventions and/or graphs of results as they come up

Also as a general reminder, it is never too early to send bullets of exciting findings, usually collected at least once a year

Talk: How does learning to write help learning to read Chinese (fMRI study) - Fan Cao
Abstract
Two types of instructions were given to a group of English speakers who learn Chinese as a L2. One is character writing and the other is pinyin writing. The hypothesis is that writing will facilitate the integration of orthographic, phonological and semantic representations by involving both perception and production and by emphasizing the special features of Chinese
characters. fMRI scans found that sensory-motor cortex and visual-spatial representation cortex are more involved if the subject had writing experience. We also found that writing training produced more elaborated representations of orthography, phonology and semantics in the brain as compared to pinyin training.

Slides to be uploaded soon

Next up: Colleen Davy will speak at the October meeting, likely the last week of Oct at CMU

Cognitive Factors Monthly Meeting Notes

2010-09-24T21:04:00Z

Ruth-Wylie: New page: Notes from the monthly meetings will be posted here, along with copies of slides and handouts. CF September 2010 September 24, 2010, LRDC

Notes from the monthly meetings will be posted here, along with copies of slides and handouts.

[[CF September 2010]] September 24, 2010, LRDC

Cognitive Factors

2010-09-24T21:02:15Z

Ruth-Wylie:

The research in this thrust is aimed at understanding cognitive learning—changes in knowledge—that result from [[instructional events]]. It builds on work in the learning sciences field at large and on research carried out in the PSLC over its first four years within the [[Refinement and Fluency]] cluster and part of the [[Coordinative Learning]] cluster, thereby merging two themes that organized the first phase of the PSLC. Each of these clusters was concerned with identifying instructional events that produce robust learning. They differed mainly in that the relevant theme within the Coordinative Learning cluster had a specific focus on instructional events that included more than one input. (A second theme within the Coordinative Learning cluster was on instructional events that provoke learning events involving more than one reasoning method and this theme will be continued in the [[Metacognition and Motivation]] thrust). In the fifth year of the PSLC, we carry forward research from each of these clusters, while making a transition to an additional set of research questions. Although we frame this section in terms of the new Cognitive Factors thrust, the research carried out during the 5th year has been initiated in the current Refinement and Fluency and in part of the Coordinative Learning clusters.

Our work on cognitive factors encompasses a triangulated set of events around learning: learning events, instructional events, and assessment events. Anything from a lesson to an entire curriculum can be considered a sequence of events whose durations vary from seconds to semesters. The hypotheses of the Cognitive Factors Thrust concern how instructional procedures (e.g., decisions about the learner’s task, materials, practice, feedback) affect learning events and thus the outcomes of learning. Learning involves the acquisition of [[knowledge components]], an increase in the [[feature validity]] and the [[strength]] of these components, and the integration of these components through practice. Our basic hypotheses include the following:

* Explicitness: Instruction that draws the learner’s attention to valid features that support the relevant knowledge components leads to more robust learning than instruction that does not.
* Assistance: The degree of assistance in the instruction affects learning in relation to student knowledge on specific knowledge components.
* Practice: Practice schedules can be optimized using models of learning based on memory activation assumptions.
* Integration: Knowledge components that are integrated during learning and practice lead to more robust learning and fluent performance across different tasks.
* [http://www.superiorpapers.com/ research papers]

The research plan tests these hypotheses across knowledge domains, as exemplified by the following projects:

''Language background factors in L2 learning''. This work illustrates the synergies that develop in the PSLC’s LearnLab context, in this case between English as a second language (ESL) director Alan Juffs and other PSLC language researchers. In a prior cluster meeting, Juffs presented ESL classroom data that compared various L1 background students in their performance on transcribing their own speech, a standard piece of instruction in the ESL curriculum. The result that caught the interest of PSLC researchers (Dunlap, Guan, Perfetti) was the very poor spelling performance of Arabic-background students, relative to Spanish, Korean, and Chinese ESL students, despite comparable levels of spoken language performance. Furthermore, Juffs identified this discrepancy as a long-standing one in ESL instruction. Although one might hypothesize that a key factor is orthographic differences between L1 and L2, this seems unlikely here. Spanish to English is closer, but Chinese to English is farther in L1-L2 orthographic similarity. The first steps toward a new study have been taken with the help of a PSLC summer intern, who coded the errors made in spelling by all L1 background learners. The pattern of errors can be characterized as qualitatively similar, differing across languages quantitatively, suggesting a generalized English spelling problem. This analysis has led to the hypothesis that feature focusing—attention to full spelling patterns—is different across the L1 backgrounds, which we will test in a training experiment that focuses attention on spelling patterns.

''Second language vocabulary learning''. Another new project originating within the Refinement and Fluency cluster will study English vocabulary learning using REAP. Based on recent research by Balass on the trade-offs between explicit (dictionary-based) and implicit (inferences from text) instruction in learning new words by monolingual subjects (Bolger et al, 2008), the new work will apply this tradeoff idea to second language learners. The hypothesis is that allowing learners to view definitions is more effective after they have read a sentence containing the word to be learned. This hypothesis reflects ideas about assistance (giving a definition versus inferring it) and the assumption that learning word meanings from context depends on the overlapping memory traces established by specific encounters with the word (Bolger et al, 2008). REAP allows us to use authentic texts for studies with students of various L1 backgrounds learning English through reading texts in their areas of interest. In our experiments, we will vary the availability of definitions provided on-line as part of the text reading.

''Explicit instruction and practice schedules in algebra and second language learning''. Foreign language learning in classrooms has stimulated research on explicit vs implicit instruction, with conclusions favoring the value of explicit instruction (Norris and Ortega, 2000). A major conclusion from PSLC work is that instruction that draws attention to critical valid features—“feature focusing”—is important in acquiring knowledge components for complex tasks. This conclusion has evidence from studies of L2 learning of the English grammar by Levin, Friskoff, Pavlik, studies of radical learning by Dunlap et al and by Pavlik, and by studies by Zhang and MacWhinney and by Liu et al on learning spoken syllables through pin-yin (alphabetic spellings). Projects in French dictation (MacWhinney) and French grammar (Presson & MacWhinney), Chinese dictation (Zhang & MacWhinney), algebra (Pavlik) and arithmetical computation (Fiez) also reflect this theme. Much of this work has been combined with completely general hypotheses about practice, based on Pavlik and Anderson (2005)’s model that describes the trade-off between the benefit of spaced practice and the cost of longer retention intervals brought by spacing. The resulting optimized practice schedule has been tested in several PSLC studies of vocabulary learning in Chinese (Pavlik, MacWhinney, Koedinger; reported in Pavlik, 2006), cues to French gender (Presson, MacWhinney, & Pavlik). Important is the generality of the optimization model. It applies to all domain content and studies in both algebra and second language learning have nee carried out. The new work in second language and in algebra builds on the synergies that have emerged from collaborations between domain researchers (e.g. MacWhinney) and Pavlik around experiments and models for optimizing practice. For Chinese, MacWhinney, Zhang, and Pavlik have developed a tutor for Chinese dictation and vocabulary learning that is being used in 18 sites. Data from these sites will be used to test the results of practice schedules and the form of instructional events (e.g. cues to gender in French) with longer term measures of robust learning. Because each of the tutors logs results to DataShop, the student records are a rich source of data for further study, including researchers beyond the PSLC.

''Learning the logic of unconfounded experiments.'' We will extend our research on college level science topics (chemistry and physics) to middle school science, with a focus on the cross-domain topic of experimental design. The ability to design unconfounded experiments and make valid inferences from their outcomes is an essential skill in scientific reasoning. The key idea here is CVS: the Control of Variables Strategy. CVS is the fundamental idea underlying the design of unconfounded experiments from which valid, causal, inferences can be made. Its acquisition is an important step in the development of scientific reasoning skills , because it provides a strong constraint on search in the space of experiments (Klahr, 2000). The Tutor for Experimental Design (TED), developed by Klahr’s research team, builds on previous work studying the different paths of learning and transfer that result from teaching CVS using different instructional methods that span from direct instruction to discovery (Chen & Klahr, 1999) and show differences along the “physical-virtual” dimension (Triona & Klahr, 2007). We build on this by constructing of a semi-autonomous tutor, then developing a full computer based tutor in Pittsburgh middle school LearnLabs and carrying out in vivo experiments with TED.

''Integration of knowledge components.'' Isolated knowledge components are not sufficient to produce fluent use of knowledge. Integrating knowledge components is important both in authentic practice that follows acquisition of knowledge components but, we hypothesize, also in the initial acquisition of components. Some of our prior work in coordinative learning establishes some of the conditions that favor multiple inputs during learning (e.g., Davenport et al in stochiometry). And experiments on fluency support the value of repeated practice in single-topic speaking as way to support fluency (de Jong, Halderman and Perfetti). In new work we propose to build on progress we have made in the study of fluency in language (de Jong et al) and arithmetic (Fiez). For example, we will follow the discovery by de Jong and colleagues that when L2 speakers repeat a speech on a single topic, their fluency scores increase on a number of measures. We will test the hypothesis that this results from the advantage of retrieving the same conceptual and lexical knowledge and overall speech plan on successive attempts, allowing fluency to increase on procedural components supported by chunking of words to phrases. We are accumulating a large database in the English LearnLab that will support the testing of additional hypotheses. The idea that some relatively simple learning (e.g. 3-5 knowledge components) is supported by integration from the beginning is being tested by Liu, Guan & Perfetti in a study of learning to read Chinese characters. The hypothesis is that when students write unfamiliar characters within the same 60-second time period that they read the character and try to learn its meaning and pronunciation, they will show more robust learning measured by reading tasks. Underlying this hypothesis is the idea that the representation of a character (or other objects that follow structural principles) can be perceptual-motor as well as visual.

== Descendents ==

To create a new project page, enclose your project name in a double set of brackets. Details for a project format may be [[ Project_Page_Template_and_Creation_Instructions | found here.]]

*[[Klahr - TED]]
*[[Perfetti - Read Write Integration]]
*[[Presson & MacWhinney - Second Language Grammar]]
*[[Davy & MacWhinney - Spanish Sentence Production]]
*[[Zhang & MacWhinney - Chinese Pinyin Learning]]
*[[Zhao & MacWhinney - English Article Usage]]
*[[Juffs - Feature Focus in Word Learning]]
*[[Fostering fluency in second language learning | de Jong - Fluency]]
*[[McLaren_-_The_Assistance_Dilemma_And_Discovery_Learning | McLaren - The Assistance Dilemma and Discovery Learning]]
*[[Wylie - Intelligent Writing Tutor]]
*[[REAP_main | Eskenazi - REAP]]
*[[Roll - Productive Failure in a Chemistry Virtual Lab]]

[[Cognitive Factors Monthly Meeting Notes]]

=== References ===
* Borek, A., McLaren, B.M., Karabinos, M., & Yaron, D. (2009). How Much Assistance is Helpful to Students in Discovery Learning? In U. Cress, V. Dimitrova, & M. Specht (Eds.), Proceedings of the Fourth European Conference on Technology Enhanced Learning, Learning in the Synergy of Multiple Disciplines (EC-TEL 2009), LNCS 5794, September/October 2009, Nice, France. (pp. 391-404). Springer-Verlag Berlin Heidelberg.

PSLC GradStudents

2010-09-07T21:37:31Z

Ruth-Wylie: /* Who are the PSLC grads? */

The purpose of this page is to serve as a repository of information relevant for grad students. We hope to maintain this page as a repository of current and relevant information for graduate students currently affiliated with the PSLC, as well as grad students who hope to be in the PSLC.

== Announcements==

1) PSLC grads are now responsible for keeping the [http://www.learnlab.org/research/wiki/index.php/PSLC_People#Graduate_Students List of PSLC Grads] up to date.

* If you know of someone who should be added (or deleted) from this list please e-mail the webmaster at bef25@pitt.edu. Alternatively, feel free to go in and update the list yourself!

2) Please e-mail Mary Lou Vercellotti ASAP if you are interested in attending the iSLC conference in Washington, D.C. on October 13-15. Up to three graduate students may attend.

3) Ultimate Block Party in Central Park, NY.

* Description: This is an outreach event for PSLC research. Faculty and graduate students are invited to attend to serve as "experts" as families visit the workshops in the park. (You will receive a brightly colored lab coat if you decide to help out.)

* How to sign up: E-mail Michael Bett at mbett@cs.cmu.edu if you are interested.

4) PSLC Graduate Student Meetings are scheduled for the following days and will begin at noon.

* Monday, September 20 in 408 LRDC - topic: grad student wiki pages
* Monday, October 18 at CMU (location tba) - topic what is the PSLC and why should you care
* Monday, November 15 in 408 LRDC - topic ?
* Monday, December 6 at CMU topic ?

== Meeting Notes==

== FAQs==

'''1. What does it take to be a PSLC grad student?'''

Well, there are basically three ways you can be considered a PSLC grad student.

a. You work on a project that receives funding from the PSLC.

b. Your advisor or collaborator receives funding from the PSLC and asks you to be involved.

c. You want to be a PSLC grad student.

'''2. What types of opportunities does the PSLC have for a grad student like me?'''

There are a variety of different levels of involvement and types of activities that the PSLC offers.

For the casual grad student, the PSLC organizes a speaker series with talks that may be of interest to students interested in the learning sciences. These are open to whomever wishes to go. There are also monthly lunch meetings where people associated with the PSLC can give a talk on their work.

The grad student community also hopes to organize events catered toward grad students, with topics like applying for grants, finding jobs, collaboration with people at other universities, etc. These are also open to the public.

For those who wish to get more involved, the grad student community also has monthly meetings to discuss center-wide issues, read and discuss articles we believe are relevant, plan future events, etc. Again, these are open to the public.

Finally, each thrust has regular or semi-regular meetings to discuss the thrust's theoretical framework, set the research agenda, and discuss the progress of projects within that thrust. While these are open to anyone, they're probably of limited interest unless you currently have or have had a project affiliated with the thrust.

'''3. What is expected of me as a PSLC grad student?'''

If you receive funding from the PSLC, you are expected, to the extent it is possible, to attend the thrust meetings for your relevant thrust, and attend the monthly PSLC lunches. The grad student community also encourages you to come to the grad student monthly meetings, of course.

If you don't receive funding from the PSLC, but still wish to be a part of the grad student community, your level of involvement is up to you.

'''How do I find out about upcoming talks/meetings/events?'''

One option is to check the Announcements section of this page. A possibly better option would be to get on our mailing list. To do that, e-mail Jo Bodnar at jobodnar AT cs.cmu.edu and ask to be put on the PSLC general mailing list and grad student mailing list.

There is also a regularly updated calendar at our [http://www.learnlab.org main webpage] that is updated regularly and gives a fairly complete account of most PSLC events.

4. '''I already consider myself a PSLC grad, and want to be included on this page! What do I have to do?'''

Well the great thing about the wiki page is that anybody can update it whenever they want! So, if you have an account here, and you know how to edit tables, you can just log in and add yourself!

The table formatting is a bit weird and hard to follow, so if you want to add yourself, the easiest thing to do is just copy this text:

<pre>
|-
| Name || University || Advisor || e-mail address || Bio || Personal Webpage || Link to PSLC project page [Project page URL Project page title]
</pre>

and paste it into the appropriate place on the table. With your own information, of course.

If you don't have an account already, you can easily request one (NOTE: I forget how to do it- I'll need to add that). Once you have an account, you can just click "Edit" above the table, and you can add yourself.

5. '''But that's such a pain! Isn't there an easier way?!'''

There sure is! If you don't want to make all that effort just to have your name and e-mail address on a page, just send your info (you could even put it in the format given above!) to our Wikimaster (yep, we made that word up!), Ben Friedline, at bef25 AT pitt.edu, and he'll put it on here.

== Who are the PSLC grads? ==

{| border=1 cellspacing="0" cellpadding="5" style="text-align: left;"
|-
! Grad Student Name
! University/Department
! Advisor
! E-mail
! Bio
! Personal Webpage
! PSLC Projects
|-
| Colleen Davy || Carnegie Mellon/Psychology || Brian MacWhinney || cdavy1@andrew.cmu.edu || I am interested in how adult second language learners develop fluent speaking skills in their second language. || N/A || [http://www.learnlab.org/research/wiki/index.php/Davy_%26_MacWhinney_-_Spanish_Sentence_Production Spanish Sentence Production]
|-
| Benjamin Friedline || University of Pittsburgh || Alan Juffs || bef25@pitt.edu || I am interested in how adult second language learners acquire morphology in a second language. || N/A || [http://www.learnlab.org/research/wiki/index.php/Juffs_-_Feature_Focus_in_Word_Learning Feature Focus in Word Learning]
|-
| Ruth Wylie || Carnegie Mellon, HCII || Ken Koedinger & Teruko Mitamura || rwylie@cs.cmu.edu || I'm interested in second language learning and self-explanation. || [http://ruthwylie.wordpress.com/ http://www.cs.cmu.edu/~rwylie] || [http://www.learnlab.org/research/wiki/index.php/Wylie_-_Intelligent_Writing_Tutor Self-Explanation and ESL]
|}

== Science of Learning Relevant Courses ==
The PIER program offers three courses -- see the [http://www.cmu.edu/pier PIER Web page]

See also the courses taught be any of the PSLC faculty.

(Please add the names of relevant courses and web pointers if possible!)

<pre>
05832 / 05432 Cognitive Modeling & Intelligent Tutoring Systems
3:00pm-4:20pm, Tuesdays and Thursdays, Fall 2010
Room 3002, Newell-Simon Hall, Carnegie Mellon University
9 units
Dr. Vincent Aleven, aleven@cs.cmu.edu
</pre>

Students in this course will learn about the Cognitive Tutor technology that has been demonstrated to dramatically enhance student learning in domains like math, science, and computer programming. This type of tutoring software is currently in use in 2,700 schools around the country and is used extensively as platform for learning sciences research. The technology is grounded in artificial intelligence, cognitive psychology, and cognitive task analysis. Students will learn data-driven and theoretical methods for analyzing human problem solving and will learn to use such data to inform the design of intelligent tutoring systems. Course projects will focus on the development of an intelligent tutor using CTAT, the Cognitive Tutor Authoring Tools (see http://ctat.pact.cs.cmu.edu). Some assignments will focus on creating cognitive models in the Jess production rule modeling language.

Students should either have programming skills, or experience in the cognitive psychology of human problem solving, or HCI / design skills, or permission from the instructor.

Wylie - Intelligent Writing Tutor

2010-09-02T15:24:59Z

Ruth-Wylie:

File:IWT study4 learninggains.jpg

2010-09-02T15:23:29Z

Ruth-Wylie:

File:IWT study4 efficiency.jpg

2010-09-02T15:23:20Z

Ruth-Wylie:

File:IWT study3 learninggains.jpg

2010-09-02T15:23:10Z

Ruth-Wylie:

File:IWT study3 efficiency.jpg

2010-09-02T15:22:55Z

Ruth-Wylie:

File:IWT study2 learninggains.jpg

2010-09-02T15:22:45Z

Ruth-Wylie:

File:IWT study2 efficiency.jpg

2010-09-02T15:22:33Z

Ruth-Wylie:

File:IWT study1 learninggains.jpg

2010-09-02T15:22:13Z

Ruth-Wylie:

Wylie - Intelligent Writing Tutor

2010-09-02T15:12:47Z

Ruth-Wylie:

Wylie - Intelligent Writing Tutor

2010-09-02T15:08:53Z

Ruth-Wylie:

Wylie - Intelligent Writing Tutor

2010-09-02T14:26:54Z

Ruth-Wylie:

{| class="wikitable" border="1" style="margin: 2em auto 2em auto"
|-
! PI
| Ruth Wylie, Ken Koedinger, Teruko Mitamura
|-
! Start date
| September 2009
|-
! End date
| May 2009
|-
! Learnlab
| English

== Extending the Self-Explanation Effect to Second Language Grammar Lewarning ==
Ruth Wylie, Ken Koedinger, and Teruko Mitamura

=== Background ===

Self-explanation is a successful instructional technique for domains like math and science but little work has investigated whether the benefits of self-explanation extend to second language learning. In this program of research, we study the effects of adding self-explanation prompts to tutors designed to teach the English article system (teaching students when to say a, an, the, or use no article at all). Our earlier studies ([[Assistance_Dilemma_English_Articles |link]]) showed that self-explanation is helpful for learning English grammar but that boundary conditions may exist. In this study, we attempt to understand the mechanisms behind the self-explanation effect.

=== Study 1 ===

==== Independent Variables ====
Students were assessed via a pre and posttest that contained items isomorphic to the article choice tutored items. In addition, a long-term retention test will be administered as part of the students' final exam.

==== Dependent Variables ====
Students were randomly assigned to one of three tutoring conditions:

Article Choice: Using the article choice tutor, students complete cloze tasks and choose the correct article (a, an, the, or no article) that completes the sentence. They receive immediate feedback on their selections and have access to hints.

Explanation Choice: The Explanation Choice tutor is a modification of the article choice condition. Instead of choosing the correct article to complete the sentence, for half of the items, students are given the sentence with the article already filled-in and asked to choose the rule or feature of the sentence that best explains the article use. For each of the 6 rules that are addressed in the tutor, students complete 10 items according to the schedule below:

2 Explanation Choice --> 1 Article Choice --> 1 Explanation Choice --> 1 Article Choice --> 1 Explanation Choice --> 1 Article Choice --> 1 Explanation Choice --> 2 Article Choice

Example Choice: The Example Choice tutor was also a modification of the Article Choice condition. For some of the items in this tutor, students were given a target sentence with the article completed and asked to choose the example sentence that used the same rule. The Example Choice tutor followed the same interleaved pattern as the Explanation Choice tutor.

==== Results ====
(November 2009) The main portion of data collection for this study is complete. However, the long-term retention measure will not be administered until December 2009.)

Wylie - Intelligent Writing Tutor

2009-12-21T17:00:27Z

Ruth-Wylie: /* Results */

{| class="wikitable" border="1" style="margin: 2em auto 2em auto"
|-
! PI
| Ruth Wylie, Ken Koedinger, Teruko Mitamura
|-
! Start date
| September 2009
|-
! End date
| May 2009
|-
! Learnlab
| English
|}

== Extending the Self-Explanation Effect to Second Language Grammar Lewarning ==
Ruth Wylie, Ken Koedinger, and Teruko Mitamura

=== Background ===

Self-explanation is a successful instructional technique for domains like math and science but little work has investigated whether the benefits of self-explanation extend to second language learning. In this program of research, we study the effects of adding self-explanation prompts to tutors designed to teach the English article system (teaching students when to say a, an, the, or use no article at all). Our earlier studies ([[Assistance_Dilemma_English_Articles |link]]) showed that self-explanation is helpful for learning English grammar but that boundary conditions may exist. In this study, we attempt to understand the mechanisms behind the self-explanation effect.

=== Study 1 ===

==== Independent Variables ====
Students were assessed via a pre and posttest that contained items isomorphic to the article choice tutored items. In addition, a long-term retention test will be administered as part of the students' final exam.

==== Dependent Variables ====
Students were randomly assigned to one of three tutoring conditions:

Article Choice: Using the article choice tutor, students complete cloze tasks and choose the correct article (a, an, the, or no article) that completes the sentence. They receive immediate feedback on their selections and have access to hints.

Explanation Choice: The Explanation Choice tutor is a modification of the article choice condition. Instead of choosing the correct article to complete the sentence, for half of the items, students are given the sentence with the article already filled-in and asked to choose the rule or feature of the sentence that best explains the article use. For each of the 6 rules that are addressed in the tutor, students complete 10 items according to the schedule below:

2 Explanation Choice --> 1 Article Choice --> 1 Explanation Choice --> 1 Article Choice --> 1 Explanation Choice --> 1 Article Choice --> 1 Explanation Choice --> 2 Article Choice

Example Choice: The Example Choice tutor was also a modification of the Article Choice condition. For some of the items in this tutor, students were given a target sentence with the article completed and asked to choose the example sentence that used the same rule. The Example Choice tutor followed the same interleaved pattern as the Explanation Choice tutor.

==== Results ====
(November 2009) The main portion of data collection for this study is complete. However, the long-term retention measure will not be administered until December 2009.)

Wylie - Intelligent Writing Tutor

2009-12-21T16:59:32Z

Ruth-Wylie: New page: {| class="wikitable" border="1" style="margin: 2em auto 2em auto" |- ! PI | Ruth Wylie, Ken Koedinger, Teruko Mitamura |- ! Start date | September 2009 |- ! End date | May 2009 |- ! Lea...

{| class="wikitable" border="1" style="margin: 2em auto 2em auto"
|-
! PI
| Ruth Wylie, Ken Koedinger, Teruko Mitamura
|-
! Start date
| September 2009
|-
! End date
| May 2009
|-
! Learnlab
| English
|}

== Extending the Self-Explanation Effect to Second Language Grammar Lewarning ==
Ruth Wylie, Ken Koedinger, and Teruko Mitamura

=== Background ===

Self-explanation is a successful instructional technique for domains like math and science but little work has investigated whether the benefits of self-explanation extend to second language learning. In this program of research, we study the effects of adding self-explanation prompts to tutors designed to teach the English article system (teaching students when to say a, an, the, or use no article at all). Our earlier studies ([[Assistance_Dilemma_English_Articles |link]]) showed that self-explanation is helpful for learning English grammar but that boundary conditions may exist. In this study, we attempt to understand the mechanisms behind the self-explanation effect.

=== Study 1 ===

==== Independent Variables ====
Students were assessed via a pre and posttest that contained items isomorphic to the article choice tutored items. In addition, a long-term retention test will be administered as part of the students' final exam.

==== Dependent Variables ====
Students were randomly assigned to one of three tutoring conditions:

Article Choice: Using the article choice tutor, students complete cloze tasks and choose the correct article (a, an, the, or no article) that completes the sentence. They receive immediate feedback on their selections and have access to hints.

Explanation Choice: The Explanation Choice tutor is a modification of the article choice condition. Instead of choosing the correct article to complete the sentence, for half of the items, students are given the sentence with the article already filled-in and asked to choose the rule or feature of the sentence that best explains the article use. For each of the 6 rules that are addressed in the tutor, students complete 10 items according to the schedule below:

2 Explanation Choice --> 1 Article Choice --> 1 Explanation Choice --> 1 Article Choice --> 1 Explanation Choice --> 1 Article Choice --> 1 Explanation Choice --> 2 Article Choice

Example Choice: The Example Choice tutor was also a modification of the Article Choice condition. For some of the items in this tutor, students were given a target sentence with the article completed and asked to choose the example sentence that used the same rule. The Example Choice tutor followed the same interleaved pattern as the Explanation Choice tutor.

==== Results ====
(November 2009) The main portion of data collection for this study is complete. However, the long-term retention measure will not be administered until December 2009.

Extending the Self-Explanation Effect to Second Language Grammar Learning

2009-11-17T19:33:01Z

{| class="wikitable" border="1" style="margin: 2em auto 2em auto"
|-
! PI
| Ruth Wylie, Ken Koedinger, Teruko Mitamura
|-
! Start date
| September 2009
|-
! End date
| May 2009
|-
! Learnlab
| English
|}

== Extending the Self-Explanation Effect to Second Language Grammar Lewarning ==
Ruth Wylie, Ken Koedinger, and Teruko Mitamura

=== Background ===

Self-explanation is a successful instructional technique for domains like math and science but little work has investigated whether the benefits of self-explanation extend to second language learning. In this program of research, we study the effects of adding self-explanation prompts to tutors designed to teach the English article system (teaching students when to say a, an, the, or use no article at all). Our earlier studies ([[Assistance_Dilemma_English_Articles |link]]) showed that self-explanation is helpful for learning English grammar but that boundary conditions may exist. In this study, we attempt to understand the mechanisms behind the self-explanation effect.

=== Study 1 ===

==== Independent Variables ====
Students were assessed via a pre and posttest that contained items isomorphic to the article choice tutored items. In addition, a long-term retention test will be administered as part of the students' final exam.

==== Dependent Variables ====
Students were randomly assigned to one of three tutoring conditions:

Article Choice: Using the article choice tutor, students complete cloze tasks and choose the correct article (a, an, the, or no article) that completes the sentence. They receive immediate feedback on their selections and have access to hints.

Explanation Choice: The Explanation Choice tutor is a modification of the article choice condition. Instead of choosing the correct article to complete the sentence, for half of the items, students are given the sentence with the article already filled-in and asked to choose the rule or feature of the sentence that best explains the article use. For each of the 6 rules that are addressed in the tutor, students complete 10 items according to the schedule below:

2 Explanation Choice --> 1 Article Choice --> 1 Explanation Choice --> 1 Article Choice --> 1 Explanation Choice --> 1 Article Choice --> 1 Explanation Choice --> 2 Article Choice

Example Choice: The Example Choice tutor was also a modification of the Article Choice condition. For some of the items in this tutor, students were given a target sentence with the article completed and asked to choose the example sentence that used the same rule. The Example Choice tutor followed the same interleaved pattern as the Explanation Choice tutor.

==== Results ====
(November 2009) The main portion of data collection for this study is complete. However, the long-term retention measure will not be administered until December 2009.

Refinement and Fluency

2009-11-17T19:09:55Z

Ruth-Wylie: /* Explicit instruction */

= The PSLC Refinement and Fluency cluster =

== Abstract ==
The studies in this cluster concern the design and organization of instructional activities to facilitate the acquisition, [[refinement]], and fluent control of critical [[knowledge components]]. The research of the cluster addresses a series of core propositions, including but not limited to the following.

1. cognitive task analysis or knowledge component analysis: Complex knowledge consists of smaller components that can be identified through analysis of knowledge-based task performance and tested in experiments. To design effective instruction, learning tasks are anlayzed into simpler task components.

2. fluency from basics: For true fluency, higher level skills must be grounded on well-practiced lower level skills.

3. scheduling of practice: [[Optimized scheduling]] of [[practice]] uses principles of memory to maximize robust learning and achieve mastery.

4. [[explicit instruction]]: Explicit instruction, i.e. instruction that either directly asserts information ("facts") or provides rules, facilitates the acquisition and refinement of specific skills. Rules are effective only when they are relatively simple.

5. [[implicit instruction]]: Implicit instruction, i.e. exposure to to-be-learned patterns, can foster the development of pattern familiarity and strengthen connections of these patterns to other patterns.

6. immediacy of feedback: A corollary of the scheduling and explicit instruction propositions is that immediate feedback facilitates learning.

7. [[cue validity]]: In both explicit and implicit instruction, the validity of a cue for a knowledge component affects the learning of that knowledge component. (Cue validity is related to [[feature validity]].)

8. [[focusing]]: Instruction that directs (focuses) the learner's attention to valid cues leads to more robust learning than unfocused instruction or instruction that focuses on less valid cues.

9. learning to learn: The acquisition of skills and strategies that can generalize across learning tasks can promote new learning. Examples may be deep analysis, help-seeking, use of advance organizers, and, most generally, meta-cognitive strategies.

10. [[transfer]]: A learner's earlier knowledge places strong constraints on new learning, promoting some forms of learning, while inhibiting others.

The overall hypothesis is that instruction that systematically reflects the complex [[features]] of targeted knowledge in relation to the learner’s existing knowledge leads to more robust learning than instruction that does not. The principle is that the gap between targeted knowledge and existing knowledge needs to be directly reflected in the organization of instructional events. This organization includes the structure of knowledge components selected for instruction, the scheduling of learning events, practice, recall opportunities, explicit and implicit presentations, and other activities.

This hypothesis can be rephrased in terms of the PSLC general hypothesis, which is that [[robust learning]] occurs when the [[learning event space]] is designed to include appropriate target paths, and when students are encouraged to take those paths. The studies in this cluster focus on the formulation of well specified target paths with highly predictable learning outcomes.

 <center>[[Image:Rf.JPG]]</center>

==Significance==
A core theme in this cluster is that instruction in basic skills can facilitate the acquisition and refinement of knowledge and prepare the learner for [[fluency]]-enhancing practice. Instruction that provides practice and feedback for basic skills on a schedule that closely matches observed student abilities is important for this goal, and can be effectively delivered by computer. In the area of second language learning, the strengths of computerized instruction are matched by certain weaknesses. In particular, computerized tutors are not yet good at speech recognition, making it difficult to assess student production. Moreover, contact with a human teacher can increase the breadth of language usage, as well as motivation. Therefore, an optimal environment for language learning would combine the strengths of computerized instruction with those of classroom instruction. It is possible that a similar analysis will apply to science and math.

== Glossary ==
[[:Category:Refinement and Fluency|Refinement and Fluency]] glossary.

== Research question ==
The overall research question is how can instruction optimally support the acquisition, refinement, and fluent use of complex targeted knowledge, taking into account the learner’s existing knowledge in relation to the knowledge demands of the target domain? In examining this general question, the studies focus on features of the learning situation, including the following: the cognitive demands of targeted knowledge components, the scheduling of practice, the timing and extent of explicit [[instructional method|instructional events]] relative to implicit learning opportunities, and the role of feedback.

== Independent variables ==
At a general level, the research varies the organization of instructional events. This organization variable is typically based on alternative analyses of task demands, relevant knowledge components, and learner background.

== Dependent variables ==
The dependent variables in these studies assess learner performance during learning events and following learning. Typical measures are percentage correct and number of learning trials or time to reach a given standard of performance. Response times are also measured in some cases.

== Hypotheses ==
Instruction that systematically reflects the complex features of targeted knowledge in relation to the learner’s existing knowledge leads to more robust learning than instruction that does not. More specifically, the initial acquisition of knowledge and its refinement benefit from instructional activities that require the learner to attend to and encode [[valid features]] of the learning content. The fluency corollary: Fluency builds on the knowledge components acquired and refined in learning, strengthening and integrating these components through practice.

Specific hypotheses about the organization of instruction derive from task analyzes of specific domain knowledge and the existing knowledge of the learner. A background assumption for most studies is that fluency is grounded in well-practiced lower level skills. A few examples of specific hypotheses are as follows:

1. Scheduling of practice hypothesis: The optimal scheduling of practice uses principles of memory consolidation to maximize robust learning and achieve mastery.

2. Resonance hypothesis: The acquisition of knowledge components can be facilitated by evoking associations between divergent coding systems. (This hypothesis is similar or perhaps the same as [[Coordinative Learning]] hypothesis or [[co-training]] more specifically whereby "divergent coding systems" here may be the same as "multiple input sources" in co-training.)

3. [[Explicit instruction]] hypothesis: Explicit rule-based instruction facilitates the acquisition of specific skills, but only if the rules are simple.

4. [[Implicit instruction]] hypothesis: Implicit instruction or exposure serves to foster the development of initial familiarity with larger patterns.

5. Feedback hypothesis: Instruction that provides immediate, diagnostic feedback will be superior to instruction that does not.

6. Cue validity hypothesis: In both explicit and implicit instruction, cue validity plays a central role in determining ease of learning of knowledge components. See also [[feature validity]].

7. [[Focusing]] hypothesis: Instruction that focuses the learner's attention on valid cues will lead to more robust learning than unfocused instruction or instruction that focuses on less valid cues.

8. Learning to learn hypothesis: The acquisition of certain skills in one context support future learning in other contexts. Such skills include problem analysis, help-seeking, or advance organizers.

9. Learner knowledge hypothesis: A learner's existing knowledge places strong constraints on new learning, promoting some forms of learning, while blocking others.

10. Active learning hypothesis: Even in simple tasks, learning is more robust when the learner actively engages in the learning material.

== Explanation ==
All knowledge involves content and procedures that are specific to a domain. An analysis of the domain reveals the complexities that a learner of a given background will face and the knowledge components that are part of the overall complexity. Accordingly, the organization of instruction is critical in allowing the learner to attend to the critical valid features of knowledge components and to integrated them in authentic performance. Acquiring valid features and strengthening their associations facilitates retrieval during subsequent assessment and instruction, leading to more robust learning. Additionally, robust learning is increased by the scheduling of learning events that promotes the [[long-term retention]] of the associations.

== Descendents ==

=== Explicit instruction ===
'''A. Explicit vs Implicit.''' These projects typically compare a more explict form of instruction with a more implict form
* [[Learning the role of radicals in reading Chinese]] (Liu et al.)
* [[Basic skills training|French dictation training]] (MacWhinney)
* [[Providing optimal support for robust learning of syntactic constructions in ESL]] (Levin, Frishkoff, De Jong, Pavlik)
* [[Extending the Self-Explanation Effect to Second Language Grammar Learning]] (Wylie, Koedinger, Mitamura)

'''B. Explicit attention manipulations''' studies typically vary features available to learner
* [[Chinese pinyin dictation]] (Zhang-MacWhinney)
* [[Learning a tonal language: Chinese]] (Wang, Perfetti, Liu) [Also Coordinative learning]
* [[French gender cues | French grammatical gender cue learning]] (Presson, MacWhinney)
** [[Learning French gender cues with prototypes | Instruction of French gender cues]] (Presson, MacWhinney)
**[[French gender prototypes | Lab study of grammar learning contrasting explicit and implicit instruction and prototype usage]] (Presson, MacWhinney)
**[[French gender attention | Lab study of effects of time pressure and explicitness on gender learning]] (Presson, MacWhinney)

'''C. Explicit instruction: Practice and Scheduling''' Typical studies control practice events and provide feedback
* [[Optimizing the practice schedule]] (Pavlik et al.) [[Applying optimal scheduling of practice in the Chinese Learnlab|1]]
* [[Japanese fluency]] (Yoshimura-MacWhinney)
* [[Fostering fluency in second language learning]] (De Jong, Halderman, Perfetti)
* [[Using learning curves to optimize problem assignment]] (Cen & Koedinger)
* [[Learning ESL Vocabulary with Context and Definitions: Order Effects and Self-Generation]] (Balass, Nelson, Perfetti)

=== Knowledge accessibility ===
'''A. Background knowledge''' These projects directly study effects of learners' background knowledge
* [[Intelligent_Writing_Tutor | First language effects on second language grammar acquisition]] (Mitamura, Wylie)
* [[Assistance_Dilemma_English_Articles | The Assistance Dilemma and the English Article System]] (Wylie, Mitamura, Koedinger)
* [[The_Help_Tutor__Roll_Aleven_McLaren|Tutoring a meta-cognitive skill: Help-seeking (Roll, Aleven & McLaren)]] [Also in Interactive Communication]
* [[The Impact of Native Writing Systems on 2nd Language Reading]] (Einikis, Ben-Yehudah, Fiez)

'''B. Availability of knowledge during learning'''
* [[Optimizing the practice schedule]] (Pavlik et al.) [[Understanding paired associate transfer effects based on shared stimulus components|2]], [[Applying optimal scheduling of practice in the Chinese Learnlab|1]], [[Understanding encoding inhibition, retrieval inhibition and destructive interference effects of errors during practice|3]]
* [[Using syntactic priming to increase robust learning]] (De Jong, Perfetti, DeKeyser)
* [[Composition_Effect__Kao_Roll|What is difficult about composite problems? (Kao, Roll)]]
* [[Arithmetical fluency project]] (Fiez)
* [[A word-experience model of Chinese character learning]] (Reichle, Perfetti, & Liu)
* [[Integrated Learning of Chinese]] (Liu, Perfetti, Wang, Wu)
* [[Integration of reading, writing and typing in learning Chinese words]] (Liu, Perfetti, Guan, Wu, Wang)

=== Active processing ===
These projects also include some addressing issues of learner control
* [[Mental rotations during vocabulary training]] (Tokowicz-Degani)
*[[Note-Taking_Technologies | Note-taking Project Page (Bauer & Koedinger)]] [Also in Coordinative Learning]
**[[Note-Taking: Restriction and Selection]] (completed)
**[[Note-Taking: Focusing On Concepts]] (planned)
**[[Note-Taking: Focusing On Quantity]] (planned)
*[[Handwriting Algebra Tutor]] (Anthony, Yang & Koedinger) [Also in Coordinative Learning]
**[[Lab study proof-of-concept for handwriting vs typing input for learning algebra equation-solving]] (completed)
**[[In vivo comparison of Cognitive Tutor Algebra using handwriting vs typing input]] (in progress)

===Other===

* [[Development of a Novel Writing System]] (Greene, Durisko, Ciuca, Fiez)

== Annotated bibliography ==
Forthcoming

[[Category:Cluster]]

Assistance Dilemma English Articles

2009-11-17T19:08:19Z

Ruth-Wylie:

{| class="wikitable" border="1" style="margin: 2em auto 2em auto"
|-
! PI
| Ruth Wylie, Teruko Mitamura, Ken Koedinger
|-
! Others with > 160 hours
| Jim Rankin
|-
! Start date study 1
| September 2008
|-
! End date study 1
| December 2008
|-
! Start date study 2
| January 2009
|-
! End date study 2
| May 2009
|-
! Learnlab
| English
|}

== Intelligent Writing Tutor ==
Ruth Wylie, Teruko Mitamura, Ken Koedinger, and Jim Rankin

=== Background ===

This project focuses on integrated theory development with respect to the Assistance Dilemma (Koedinger and Aleven, 2007), specifically the instructional principle of self-explanation. We explore whether having students self-explain and thereby adding difficulty aids or harms eventual learning outcomes.

This work sets out to test, for the first time, whether the success of self-explanation extend to second language grammar learning, particularly the English article system (teaching students the difference between when to say “a dog” vs “the dog”). Previous work has demonstrated the effectiveness of self-explanation to enhance robust learning across multiple math and science domains and a variety of learners. Our goal is to better understand how, why, and when self-explanation works by investigating the effects of self-explanation on second language acquisition.

=== Self-Explanation Study 1 ===

==== Hypothesis ====
Since the added process of self-explanation is generative, students in the self-explanation with menu condition will show greater learning gains on both normal and robust learning measures than those in the practice-only condition and free-form self-explanation condition. We hypothesize that the added difficulty of generating explanations in the free-form condition is extraneous and thus will result in less learning than the menu-based self-explanation conditions.

Alternatively, generating self-explanations may require too much time and thus it may be better to provide students with extra examples and to implicitly learn the rules through the practice-only condition.

In addition, while self-explanation has proved to lead to greater learning in other domains, this would be the one of the first, if not the first, study to empirically examine its effects in second language grammar acquisition.

==== Independent Variables ====
[[Image:Picture 1.jpg|600px]] 
Image 1: Practice-Only Tutor - Using this tutor, students select an article (a, an, the, no article) from each sentence to complete the sentence.

[[Image:Picture 7.jpg|600px]] 
Image 2: Menu-based Self-Explanation Tutor - Using this tutor, students select an article (a, an, the, no article) from each sentence and choose the feature of the sentence that best explains the article choice.

[[Image:Picture 2.jpg|600px]] 
Image 3: Free-form Self-Explanation Tutor- Using this tutor, students select an article (a, an, the, no article) from each sentence and write an explanation for why they made their choice.

==== Dependent Variables ====
All students completed a computer-based pre and posttest that consisted of article-only and article with explanation items. In the article-only items, students chose an article from a dropdown menu to complete the sentence. In the article with explanation items, students chose an article to complete the sentence and then chose the feature or rule that explained their answer. No hints were available during the tests, and students did not receive feedback on their answers.

==== Results ====
Students (n=63) in all conditions showed significant pre to posttest improvement. While there were no significant main effects of condition, there was a significant aptitude-treatment interaction (F(2, 60) = 3.54, p = 0.036) of theoretical interest. The dependent variable in this analysis was a learning efficiency score, which is a measure that combines learning gains and time-on-task into a single construct, and the article pre-test was the aptitude measure. As shown in the figure below, lower aptitude students benefited more (i.e., learned more in less time) from the self-explanation manipulations whereas higher aptitude students benefited more from the practice-only condition. This interaction is consistent with the theoretical notion that [[self-explanation]] is beneficial primarily during early construction and [[refinement]] of [[knowledge components]] whereas practice alone is better during knowledge [[strength|strengthening]] and [[fluency]] development. It is also consistent with research on the [[worked example principle]] and particularly the "expertise reversal effect" (Kalyuga, Chandler, Tuovinen, & Sweller, 2001).

[[Image:Efficiency.jpg]] 

=== Self Explanation Study 2 ===
The second study, conducted in Spring 2009 in Levels 3, 4, and 5 of the English LearnLab compared a practice-only tutoring system to a worked examples with self-explanation system. The goal of this study was to further investigate the effects of self-explanation on second language grammar learning. In the self-explanation conditions in the previous study, students were required to select an article and select an explanation. In this study, we isolated the two tasks such that students were either selecting articles (practice-only) or selecting an exlanation (worked examples with self-explanation).

====Independent Variables====

[[Image:Practice_only.jpg]] 
Practice-Only Tutor: Working with the practice-only tutor, students select an article (a, an, the, or no article) from a drop-down. Students receive immediate feedback on their selections and can request hints to help choose the right answer.

[[Image:Worked_ex_with_SE.jpg]] 
Worked Example with Self-Explanation Tutor:In the worked example plus self-explanation tutor, students are given a completed sentence with the target article highlighted and are asked to select the feature of the sentence that best explains the article choice.

====Dependent Variables====
Similar to Study 1, students were given a pretest and immediate posttest with items similar to those on which they were tutored. The pre and posttests contained both article selection and explanation selection items.

====Results====
Results showed that students (n=101) in both conditions showed significant learning gains (F(1, 99) = 40.28, p < 0.001) but there was no difference between conditions.

[[Image:Learning_gain.jpg]]

=== Contributions ===
These studies are the first to experimentally test the effects of self-explanation in second language grammar learning, and the current results suggest that there are limitations to the benefits of self-explanation. This work highlights the need to continue investigating the self-explanation effect in new and different domains. It suggests there may be limitations to its applicability. Additionally, it is important to understand the source of difficulty within a domain and identify how self-explanations may or may not address it. More research is needed to further specify those boundary conditions and relate them to basic understanding of cognitive processes.

=== References ===
1. Bjork, R. A. (1994). Memory and metamemory considerations in the training of human beings. In J. Metcalfe and A. Shimamura (Eds.), Metacognition: Knowing about knowing (pp.185-205). Cambridge, MA: MIT Press.
 2. Celce-Murcia, Marianne, and Larsen-Freeman, Diane. (1983). The Grammar Book: An ESL/EFL Teacher’s Course. Rowley, Massachusetts: Newbury House Publishers.
 3. Elson, Allegra B. Fossilized language forms: Implications and a search for solutions in an adult English as second language classroom. A PALPIN Inquiry Project, 2004.
 4. Ericsson, K. A, & Simon, H. (1984). Protocol analysis: Verbal reports as data. Cambridge, MA: MIT Press
 5. Hawkins, J. A. (1991). On (in) definite articles: implicatures and (un) grammaticality prediction. Journal of Linguistics(27), 405-442.
 6. Knowledge Component Hypothesis (2007, April 30). In PSLC Theory Wiki. Retrieved March 27, 2008, from http://learnlab.org/research/wiki/index.php/Knowledge_component_hypothesis
 7. Koedinger, K., & Aleven, V. (2007). Exploring the assistance dilemma in experiments with Cognitive Tutors. Educational Psychology Review. 19(3) 239-264.
 8. Level 4 Writing Objectives. (2007, Novermber 13). Retrieved March 19, 2008, from http://learnlab.org/learnlabs/english/Level_4_Objectives/Objectives_Wr4fin.dco
 9. Liu, D. & Gleason, J. (2002). Acquisition of the Article THE by Nonnative Speakers of English, An Analysis of Four Nongeneric Uses. SSLA, 24, 1-26.
 10. Master, P. (1997). The English Article System: Acquisition, Function, and Pedagogy. System. 25,(2) 215-232.
 11. Pavlik Jr., P.I., and Anderson, J. R. (2005). Practice and forgetting effects on vocabulary memory: An activation-based model of the spacing effect. Cognitive Science, 29(4), 559-586.
 12. Schooler, JW & Engstler-Schooler, TY. (1990). Verbal overshadowing of visual memories: some things are better left unsaid. Cognitive Psychology. 22(1):36-71.
 13. VanLehn, K., Siler, S., Murray, C., Yamauchi, T., & Baggett, W.B. (2003). Why do only some events cause learning during human tutoring? Cognition and Instruction, 21(3), 209-249.
 14. Wylie, R. (2007) Are we asking the right questions? Understanding which tasks lead to the robust learning of English grammar. Accepted as a Young Researchers Track paper at the 13th International Conference on Artificial Intelligence in Education. Marina del Rey, California. July 9 – 13, 2007.
 15. Wylie, R. (2008) Putting a/the stake in the ground: Making a priori predictions of student learning. Accepted as a Young Researchers Track paper at Intelligent Tutoring Systems 2008. Montreal, Canada. June 23 – 27, 2008.

Assistance Dilemma English Articles

2009-05-18T05:21:57Z

Ruth-Wylie: /* Research Questions */

{| class="wikitable" border="1" style="margin: 2em auto 2em auto"
|-
! PI
| Ruth Wylie, Teruko Mitamura, Ken Koedinger
|-
! Others with > 160 hours
| Jim Rankin
|-
! Start date study 1
| September 2008
|-
! End date study 1
| December 2008
|-
! Start date study 2
| January 2008
|-
! End date study 2
| May 2008
|-
! Learnlab
| English
|}

== Intelligent Writing Tutor ==
Ruth Wylie, Teruko Mitamura, Ken Koedinger, and Jim Rankin

=== Background ===

This project focuses on integrated theory development with respect to the Assistance Dilemma (Koedinger and Aleven, 2007), specifically the instructional principle of self-explanation. We explore whether having students self-explain and thereby adding difficulty aids or harms eventual learning outcomes.

This work sets out to test, for the first time, whether the success of self-explanation extend to second language grammar learning, particularly the English article system (teaching students the difference between when to say “a dog” vs “the dog”). Previous work has demonstrated the effectiveness of self-explanation to enhance robust learning across multiple math and science domains and a variety of learners. Our goal is to better understand how, why, and when self-explanation works by investigating the effects of self-explanation on second language acquisition.

=== Self-Explanation Study 1 ===

==== Hypothesis ====
Since the added process of self-explanation is generative, students in the self-explanation with menu condition will show greater learning gains on both normal and robust learning measures than those in the practice-only condition and free-form self-explanation condition. We hypothesize that the added difficulty of generating explanations in the free-form condition is extraneous and thus will result in less learning than the menu-based self-explanation conditions.

Alternatively, generating self-explanations may require too much time and thus it may be better to provide students with extra examples and to implicitly learn the rules through the practice-only condition.

In addition, while self-explanation has proved to lead to greater learning in other domains, this would be the one of the first, if not the first, study to empirically examine its effects in second language grammar acquisition.

==== Independent Variables ====
[[Image:Picture 1.jpg|600px]] 
Image 1: Practice-Only Tutor - Using this tutor, students select an article (a, an, the, no article) from each sentence to complete the sentence.

[[Image:Picture 7.jpg|600px]] 
Image 2: Menu-based Self-Explanation Tutor - Using this tutor, students select an article (a, an, the, no article) from each sentence and choose the feature of the sentence that best explains the article choice.

[[Image:Picture 2.jpg|600px]] 
Image 3: Free-form Self-Explanation Tutor- Using this tutor, students select an article (a, an, the, no article) from each sentence and write an explanation for why they made their choice.

==== Dependent Variables ====
All students completed a computer-based pre and posttest that consisted of article-only and article with explanation items. In the article-only items, students chose an article from a dropdown menu to complete the sentence. In the article with explanation items, students chose an article to complete the sentence and then chose the feature or rule that explained their answer. No hints were available during the tests, and students did not receive feedback on their answers.

==== Results ====
Students (n=63) in all conditions showed significant pre to posttest improvement; however, there was no significant differences between conditions. Analysis using efficiency scores (a measure that combines learning gains and time-on-task into a single construct) revealed a significant aptitude by treatment interaction (F(2, 60) = 3.54, p = 0.036)

[[Image:Efficiency.jpg]] 

=== Self Explanation Study 2 ===
The second study, conducted in Spring 2009 in Levels 3, 4, and 5 of the English LearnLab compared a practice-only tutoring system to a worked examples with self-explanation system. The goal of this study was to further investigate the effects of self-explanation on second language grammar learning. In the self-explanation conditions in the previous study, students were required to select an article and select an explanation. In this study, we isolated the two tasks such that students were either selecting articles (practice-only) or selecting an exlanation (worked examples with self-explanation).

====Independent Variables====

[[Image:Practice_only.jpg]] 
Practice-Only Tutor: Working with the practice-only tutor, students select an article (a, an, the, or no article) from a drop-down. Students receive immediate feedback on their selections and can request hints to help choose the right answer.

[[Image:Worked_ex_with_SE.jpg]] 
Worked Example with Self-Explanation Tutor:In the worked example plus self-explanation tutor, students are given a completed sentence with the target article highlighted and are asked to select the feature of the sentence that best explains the article choice.

====Dependent Variables====
Similar to Study 1, students were given a pretest and immediate posttest with items similar to those on which they were tutored. The pre and posttests contained both article selection and explanation selection items.

====Results====
Results showed that students (n=101) in both conditions showed significant learning gains (F(1, 99) = 40.28, p < 0.001) but there was no difference between conditions.

[[Image:Learning_gain.jpg]]

=== Contributions ===
These studies are the first to experimentally test the effects of self-explanation in second language grammar learning, and the current results suggest that there are limitations to the benefits of self-explanation. This work highlights the need to continue investigating the self-explanation effect in new and different domains. It suggests there may be limitations to its applicability. Additionally, it is important to understand the source of difficulty within a domain and identify how self-explanations may or may not address it. More research is needed to further specify those boundary conditions and relate them to basic understanding of cognitive processes.

=== References ===
1. Bjork, R. A. (1994). Memory and metamemory considerations in the training of human beings. In J. Metcalfe and A. Shimamura (Eds.), Metacognition: Knowing about knowing (pp.185-205). Cambridge, MA: MIT Press.
 2. Celce-Murcia, Marianne, and Larsen-Freeman, Diane. (1983). The Grammar Book: An ESL/EFL Teacher’s Course. Rowley, Massachusetts: Newbury House Publishers.
 3. Elson, Allegra B. Fossilized language forms: Implications and a search for solutions in an adult English as second language classroom. A PALPIN Inquiry Project, 2004.
 4. Ericsson, K. A, & Simon, H. (1984). Protocol analysis: Verbal reports as data. Cambridge, MA: MIT Press
 5. Hawkins, J. A. (1991). On (in) definite articles: implicatures and (un) grammaticality prediction. Journal of Linguistics(27), 405-442.
 6. Knowledge Component Hypothesis (2007, April 30). In PSLC Theory Wiki. Retrieved March 27, 2008, from http://learnlab.org/research/wiki/index.php/Knowledge_component_hypothesis
 7. Koedinger, K., & Aleven, V. (2007). Exploring the assistance dilemma in experiments with Cognitive Tutors. Educational Psychology Review. 19(3) 239-264.
 8. Level 4 Writing Objectives. (2007, Novermber 13). Retrieved March 19, 2008, from http://learnlab.org/learnlabs/english/Level_4_Objectives/Objectives_Wr4fin.dco
 9. Liu, D. & Gleason, J. (2002). Acquisition of the Article THE by Nonnative Speakers of English, An Analysis of Four Nongeneric Uses. SSLA, 24, 1-26.
 10. Master, P. (1997). The English Article System: Acquisition, Function, and Pedagogy. System. 25,(2) 215-232.
 11. Pavlik Jr., P.I., and Anderson, J. R. (2005). Practice and forgetting effects on vocabulary memory: An activation-based model of the spacing effect. Cognitive Science, 29(4), 559-586.
 12. Schooler, JW & Engstler-Schooler, TY. (1990). Verbal overshadowing of visual memories: some things are better left unsaid. Cognitive Psychology. 22(1):36-71.
 13. VanLehn, K., Siler, S., Murray, C., Yamauchi, T., & Baggett, W.B. (2003). Why do only some events cause learning during human tutoring? Cognition and Instruction, 21(3), 209-249.
 14. Wylie, R. (2007) Are we asking the right questions? Understanding which tasks lead to the robust learning of English grammar. Accepted as a Young Researchers Track paper at the 13th International Conference on Artificial Intelligence in Education. Marina del Rey, California. July 9 – 13, 2007.
 15. Wylie, R. (2008) Putting a/the stake in the ground: Making a priori predictions of student learning. Accepted as a Young Researchers Track paper at Intelligent Tutoring Systems 2008. Montreal, Canada. June 23 – 27, 2008.

Assistance Dilemma English Articles

2009-05-18T05:21:27Z

Ruth-Wylie:

{| class="wikitable" border="1" style="margin: 2em auto 2em auto"
|-
! PI
| Ruth Wylie, Teruko Mitamura, Ken Koedinger
|-
! Others with > 160 hours
| Jim Rankin
|-
! Start date study 1
| September 2008
|-
! End date study 1
| December 2008
|-
! Start date study 2
| January 2008
|-
! End date study 2
| May 2008
|-
! Learnlab
| English
|}

== Intelligent Writing Tutor ==
Ruth Wylie, Teruko Mitamura, Ken Koedinger, and Jim Rankin

=== Background ===

This project focuses on integrated theory development with respect to the Assistance Dilemma (Koedinger and Aleven, 2007), specifically the instructional principle of self-explanation. We explore whether having students self-explain and thereby adding difficulty aids or harms eventual learning outcomes.

This work sets out to test, for the first time, whether the success of self-explanation extend to second language grammar learning, particularly the English article system (teaching students the difference between when to say “a dog” vs “the dog”). Previous work has demonstrated the effectiveness of self-explanation to enhance robust learning across multiple math and science domains and a variety of learners. Our goal is to better understand how, why, and when self-explanation works by investigating the effects of self-explanation on second language acquisition.

=== Self-Explanation Study 1 ===
==== Research Questions ====
==== Hypothesis ====
Since the added process of self-explanation is generative, students in the self-explanation with menu condition will show greater learning gains on both normal and robust learning measures than those in the practice-only condition and free-form self-explanation condition. We hypothesize that the added difficulty of generating explanations in the free-form condition is extraneous and thus will result in less learning than the menu-based self-explanation conditions.

Alternatively, generating self-explanations may require too much time and thus it may be better to provide students with extra examples and to implicitly learn the rules through the practice-only condition.

In addition, while self-explanation has proved to lead to greater learning in other domains, this would be the one of the first, if not the first, study to empirically examine its effects in second language grammar acquisition.

==== Independent Variables ====
[[Image:Picture 1.jpg|600px]] 
Image 1: Practice-Only Tutor - Using this tutor, students select an article (a, an, the, no article) from each sentence to complete the sentence.

[[Image:Picture 7.jpg|600px]] 
Image 2: Menu-based Self-Explanation Tutor - Using this tutor, students select an article (a, an, the, no article) from each sentence and choose the feature of the sentence that best explains the article choice.

[[Image:Picture 2.jpg|600px]] 
Image 3: Free-form Self-Explanation Tutor- Using this tutor, students select an article (a, an, the, no article) from each sentence and write an explanation for why they made their choice.

==== Dependent Variables ====
All students completed a computer-based pre and posttest that consisted of article-only and article with explanation items. In the article-only items, students chose an article from a dropdown menu to complete the sentence. In the article with explanation items, students chose an article to complete the sentence and then chose the feature or rule that explained their answer. No hints were available during the tests, and students did not receive feedback on their answers.

==== Results ====
Students (n=63) in all conditions showed significant pre to posttest improvement; however, there was no significant differences between conditions. Analysis using efficiency scores (a measure that combines learning gains and time-on-task into a single construct) revealed a significant aptitude by treatment interaction (F(2, 60) = 3.54, p = 0.036)

[[Image:Efficiency.jpg]] 

=== Self Explanation Study 2 ===
The second study, conducted in Spring 2009 in Levels 3, 4, and 5 of the English LearnLab compared a practice-only tutoring system to a worked examples with self-explanation system. The goal of this study was to further investigate the effects of self-explanation on second language grammar learning. In the self-explanation conditions in the previous study, students were required to select an article and select an explanation. In this study, we isolated the two tasks such that students were either selecting articles (practice-only) or selecting an exlanation (worked examples with self-explanation).

====Independent Variables====

[[Image:Practice_only.jpg]] 
Practice-Only Tutor: Working with the practice-only tutor, students select an article (a, an, the, or no article) from a drop-down. Students receive immediate feedback on their selections and can request hints to help choose the right answer.

[[Image:Worked_ex_with_SE.jpg]] 
Worked Example with Self-Explanation Tutor:In the worked example plus self-explanation tutor, students are given a completed sentence with the target article highlighted and are asked to select the feature of the sentence that best explains the article choice.

====Dependent Variables====
Similar to Study 1, students were given a pretest and immediate posttest with items similar to those on which they were tutored. The pre and posttests contained both article selection and explanation selection items.

====Results====
Results showed that students (n=101) in both conditions showed significant learning gains (F(1, 99) = 40.28, p < 0.001) but there was no difference between conditions.

[[Image:Learning_gain.jpg]]

=== Contributions ===
These studies are the first to experimentally test the effects of self-explanation in second language grammar learning, and the current results suggest that there are limitations to the benefits of self-explanation. This work highlights the need to continue investigating the self-explanation effect in new and different domains. It suggests there may be limitations to its applicability. Additionally, it is important to understand the source of difficulty within a domain and identify how self-explanations may or may not address it. More research is needed to further specify those boundary conditions and relate them to basic understanding of cognitive processes.

=== References ===
1. Bjork, R. A. (1994). Memory and metamemory considerations in the training of human beings. In J. Metcalfe and A. Shimamura (Eds.), Metacognition: Knowing about knowing (pp.185-205). Cambridge, MA: MIT Press.
 2. Celce-Murcia, Marianne, and Larsen-Freeman, Diane. (1983). The Grammar Book: An ESL/EFL Teacher’s Course. Rowley, Massachusetts: Newbury House Publishers.
 3. Elson, Allegra B. Fossilized language forms: Implications and a search for solutions in an adult English as second language classroom. A PALPIN Inquiry Project, 2004.
 4. Ericsson, K. A, & Simon, H. (1984). Protocol analysis: Verbal reports as data. Cambridge, MA: MIT Press
 5. Hawkins, J. A. (1991). On (in) definite articles: implicatures and (un) grammaticality prediction. Journal of Linguistics(27), 405-442.
 6. Knowledge Component Hypothesis (2007, April 30). In PSLC Theory Wiki. Retrieved March 27, 2008, from http://learnlab.org/research/wiki/index.php/Knowledge_component_hypothesis
 7. Koedinger, K., & Aleven, V. (2007). Exploring the assistance dilemma in experiments with Cognitive Tutors. Educational Psychology Review. 19(3) 239-264.
 8. Level 4 Writing Objectives. (2007, Novermber 13). Retrieved March 19, 2008, from http://learnlab.org/learnlabs/english/Level_4_Objectives/Objectives_Wr4fin.dco
 9. Liu, D. & Gleason, J. (2002). Acquisition of the Article THE by Nonnative Speakers of English, An Analysis of Four Nongeneric Uses. SSLA, 24, 1-26.
 10. Master, P. (1997). The English Article System: Acquisition, Function, and Pedagogy. System. 25,(2) 215-232.
 11. Pavlik Jr., P.I., and Anderson, J. R. (2005). Practice and forgetting effects on vocabulary memory: An activation-based model of the spacing effect. Cognitive Science, 29(4), 559-586.
 12. Schooler, JW & Engstler-Schooler, TY. (1990). Verbal overshadowing of visual memories: some things are better left unsaid. Cognitive Psychology. 22(1):36-71.
 13. VanLehn, K., Siler, S., Murray, C., Yamauchi, T., & Baggett, W.B. (2003). Why do only some events cause learning during human tutoring? Cognition and Instruction, 21(3), 209-249.
 14. Wylie, R. (2007) Are we asking the right questions? Understanding which tasks lead to the robust learning of English grammar. Accepted as a Young Researchers Track paper at the 13th International Conference on Artificial Intelligence in Education. Marina del Rey, California. July 9 – 13, 2007.
 15. Wylie, R. (2008) Putting a/the stake in the ground: Making a priori predictions of student learning. Accepted as a Young Researchers Track paper at Intelligent Tutoring Systems 2008. Montreal, Canada. June 23 – 27, 2008.