https://learnlab.org/wiki/index.php?title=Koedinger_-_Toward_a_model_of_accelerated_future_learning&feed=atom&action=historyKoedinger - Toward a model of accelerated future learning - Revision history2024-03-29T13:03:59ZRevision history for this page on the wikiMediaWiki 1.31.12https://learnlab.org/wiki/index.php?title=Koedinger_-_Toward_a_model_of_accelerated_future_learning&diff=11057&oldid=prevKoedinger at 13:24, 14 September 20102010-09-14T13:24:13Z<p></p>
<table class="diff diff-contentalign-left" data-mw="interface">
<col class="diff-marker" />
<col class="diff-content" />
<col class="diff-marker" />
<col class="diff-content" />
<tr class="diff-title" lang="en">
<td colspan="2" style="background-color: #fff; color: #222; text-align: center;">← Older revision</td>
<td colspan="2" style="background-color: #fff; color: #222; text-align: center;">Revision as of 13:24, 14 September 2010</td>
</tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l1" >Line 1:</td>
<td colspan="2" class="diff-lineno">Line 1:</td></tr>
<tr><td class='diff-marker'>−</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>== Project Overview ==</div></td><td class='diff-marker'>+</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins class="diffchange diffchange-inline">=</ins>== Project Overview <ins class="diffchange diffchange-inline">=</ins>==</div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del class="diffchange diffchange-inline">This project will address goal 1 </del>of the <del class="diffchange diffchange-inline">CMDM thrust and in particular use DataShop datasets (90 in 5 years) to produce better cognitive models and verify the models with in vivo experiments</del>.  <del class="diffchange diffchange-inline">Cognitive models drive the great many instructional decisions that automated tutoring currently make, whether it is how to organize instructional messages</del>, <del class="diffchange diffchange-inline">sequence topics </del>and <del class="diffchange diffchange-inline">problems in a curriculum</del>, <del class="diffchange diffchange-inline">adapt pacing </del>to <del class="diffchange diffchange-inline">student needs, or select appropriate materials and tasks to adapt to student needs</del>.  <del class="diffchange diffchange-inline">Cognitive models also appear critical to accurate assessment of self-regulated </del>learning <del class="diffchange diffchange-inline">skills or motivational states.</del></div></td><td class='diff-marker'>+</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins class="diffchange diffchange-inline">Perhaps the most interesting </ins>of the <ins class="diffchange diffchange-inline">PSLC measures of robust learning is accelerated future learning</ins>.  <ins class="diffchange diffchange-inline">A growing number of studies</ins>, <ins class="diffchange diffchange-inline">within PSLC </ins>and <ins class="diffchange diffchange-inline">without</ins>, <ins class="diffchange diffchange-inline">have experimentally demonstrated that some instructional treatments lead </ins>to <ins class="diffchange diffchange-inline">accelerated future learning</ins>.  <ins class="diffchange diffchange-inline">These treatments (and associated studies) include inventing for future </ins>learning (<ins class="diffchange diffchange-inline">Schwartz; Roll</ins>)<ins class="diffchange diffchange-inline">, self</ins>-<ins class="diffchange diffchange-inline">explanation </ins>(<ins class="diffchange diffchange-inline">Hausmann & VanLehn</ins>), and <ins class="diffchange diffchange-inline">feature prerequisite drill (Pavlik)</ins>. <ins class="diffchange diffchange-inline"> While results are starting </ins>to <ins class="diffchange diffchange-inline">accumulate, we have little by way of precise understanding of </ins>the <ins class="diffchange diffchange-inline">learning mechanisms that yield these results</ins>. <ins class="diffchange diffchange-inline"> </ins></div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del class="diffchange diffchange-inline">Multiple algorithms have been developed for automated discovery of the attributes or factors that make up a cognitive model </del>(<del class="diffchange diffchange-inline">or a "Q matrix"</del>) <del class="diffchange diffchange-inline">including various Q</del>-<del class="diffchange diffchange-inline">matrix discovery algorithms like Rule Spaces, Knowledge Spaces, Learning Factors Analysis </del>(<del class="diffchange diffchange-inline">LFA</del>), and <del class="diffchange diffchange-inline">Bayesian exponential-family PCA</del>. <del class="diffchange diffchange-inline">This project will create an infrastructure for automatically applying such algorithms </del>to <del class="diffchange diffchange-inline">data sets in </del>the <del class="diffchange diffchange-inline">DataShop, discovering better cognitive models, and evaluating whether such models improve tutors</del>.</div></td><td colspan="2"> </td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'>−</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del class="diffchange diffchange-inline">== Planned accomplishments for PSLC Year 6 ==</del></div></td><td class='diff-marker'>+</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins class="diffchange diffchange-inline">The key goal of this project is to combine data mining </ins>and <ins class="diffchange diffchange-inline">machine learning to create a computational models of learning mechanisms that yield accelerated future learning</ins>.  We will <ins class="diffchange diffchange-inline">are fitting this modelsuch models and ablated (or “lesioned”) alternatives against relevant data to isolate critical features of </ins>the <ins class="diffchange diffchange-inline">mechanisms (e.g., Matsuda et.al</ins>, <ins class="diffchange diffchange-inline">2007</ins>, <ins class="diffchange diffchange-inline">2008) </ins>of <ins class="diffchange diffchange-inline">future </ins>learning <ins class="diffchange diffchange-inline"> (e</ins>.<ins class="diffchange diffchange-inline">g., Li, Cohen, & Koedinger, 2010; Matsuda et</ins>.<ins class="diffchange diffchange-inline">al, 2007, 2008; Shih et al., 2008</ins>)<ins class="diffchange diffchange-inline">.  We will are considering </ins>at least <ins class="diffchange diffchange-inline">three </ins>two <ins class="diffchange diffchange-inline">kinds of data sources and phenomenon.  One data source is the DataShop data associated with experiments, like those listed above, where an accelerated future learning result has been achieved.  A second data source is any DataShop data set with valid pre-and post-test data by which we can determine differences in student learning rate.  Another A third data source is any </ins>DataShop data <ins class="diffchange diffchange-inline">set with a quality knowledge component model and learning curves</ins>.  <ins class="diffchange diffchange-inline">For such a data source, we </ins>will <ins class="diffchange diffchange-inline">are creatinge statistical models of individual differences across in students in learning rate.  Dividing students into fast learners </ins>and <ins class="diffchange diffchange-inline">slow learners, we can then are testing alternative versions of the computational </ins>or <ins class="diffchange diffchange-inline">statistical models to see which best fits both the learning rate</ins>, <ins class="diffchange diffchange-inline">and perhaps error patterns</ins>, <ins class="diffchange diffchange-inline">of both slow learners </ins>and <ins class="diffchange diffchange-inline">fast learners.  In cases where we have measures of differences in students’ conceptual prerequisite knowledge </ins>(<ins class="diffchange diffchange-inline">e</ins>.<ins class="diffchange diffchange-inline">g</ins>.<ins class="diffchange diffchange-inline">, Booth’s equation solving data or Pavlik’s Chinese radical/character and pre-algebra </ins>data<ins class="diffchange diffchange-inline">)</ins>, <ins class="diffchange diffchange-inline">we can use such data </ins>to <ins class="diffchange diffchange-inline">further constrain </ins>the <ins class="diffchange diffchange-inline">computational modeling effort</ins>.  </div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del class="diffchange diffchange-inline">1. Develop code </del>and <del class="diffchange diffchange-inline">human-computer interfaces for applying, comparing and interpreting cognitive model discovery algorithms across multiple data sets in DataShop</del>.  We will <del class="diffchange diffchange-inline">document processes for how </del>the <del class="diffchange diffchange-inline">algorithms</del>, <del class="diffchange diffchange-inline">like LFA</del>, <del class="diffchange diffchange-inline">combine automation and human input to discover or improve cognitive models </del>of <del class="diffchange diffchange-inline">specific </del>learning <del class="diffchange diffchange-inline">domains</del>.  </div></td><td colspan="2"> </td></tr>
<tr><td class='diff-marker'>−</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del class="diffchange diffchange-inline">2</del>. <del class="diffchange diffchange-inline">Demonstrate the use of the model discovery infrastructure (#1</del>) <del class="diffchange diffchange-inline">for </del>at least two <del class="diffchange diffchange-inline">discovery algorithms applied to at least 4 </del>DataShop data <del class="diffchange diffchange-inline">sets</del>.  <del class="diffchange diffchange-inline">We </del>will <del class="diffchange diffchange-inline">target at least one math (Geometry area </del>and<del class="diffchange diffchange-inline">/</del>or <del class="diffchange diffchange-inline">Algebra equation solving)</del>, <del class="diffchange diffchange-inline">one science (Physics kinematics)</del>, and <del class="diffchange diffchange-inline">one language </del>(<del class="diffchange diffchange-inline">English articles) domain</del>.  </div></td><td colspan="2"> </td></tr>
<tr><td class='diff-marker'>−</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del class="diffchange diffchange-inline">3</del>. <del class="diffchange diffchange-inline">For at least one of these </del>data <del class="diffchange diffchange-inline">sets</del>, <del class="diffchange diffchange-inline">work with associated researchers </del>to <del class="diffchange diffchange-inline">perform a “close </del>the <del class="diffchange diffchange-inline">loop” experiment whereby we test whether a better cognitive model leads to better or more efficient student learning</del>.  </div></td><td colspan="2"> </td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'>−</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del class="diffchange diffchange-inline">== Integrated Research Results ==</del></div></td><td class='diff-marker'>+</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins class="diffchange diffchange-inline">A computational model of accelerated future learning </ins>that <ins class="diffchange diffchange-inline">fits a variety </ins>of <ins class="diffchange diffchange-inline">student learning data sets across </ins>math, science, and language <ins class="diffchange diffchange-inline">domains </ins>would be <ins class="diffchange diffchange-inline">a significant </ins>achievement in <ins class="diffchange diffchange-inline">theoretical integration within </ins>the learning <ins class="diffchange diffchange-inline">sciences</ins>.  </div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del class="diffchange diffchange-inline">Establishing </del>that <del class="diffchange diffchange-inline">cognitive models </del>of <del class="diffchange diffchange-inline">academic domain knowledge in </del>math, science, and language <del class="diffchange diffchange-inline">can be discovered from data </del>would be <del class="diffchange diffchange-inline">an important scientific </del>achievement<del class="diffchange diffchange-inline">.  The achievement will be greater to the extent that the discovered models involve deep or integrative knowledge components not directly apparent in surface task structure (e.g., model discovery </del>in the <del class="diffchange diffchange-inline">Geometry area domain isolated a problem decomposition skill).  The statistical model structure of competing discovery algorithms promises to shed new light on the nature or extent of regularities or laws of learning, like the power or exponential shape of learning curves, whether the complexity of task behavior is due to human or domain characteristics (the ant on the beach question), whether or not there are systematic individual differences in student </del>learning <del class="diffchange diffchange-inline">rates</del>. <del class="diffchange diffchange-inline"> </del></div></td><td colspan="2"> </td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'>−</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>== <del class="diffchange diffchange-inline">Year 6 </del>Project <del class="diffchange diffchange-inline">Deliverables </del>==</div></td><td class='diff-marker'>+</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>==<ins class="diffchange diffchange-inline">= </ins>Project <ins class="diffchange diffchange-inline">Goals=</ins>==</div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>* <del class="diffchange diffchange-inline">Develop code </del>and <del class="diffchange diffchange-inline">human-computer interfaces for applying, comparing </del>and <del class="diffchange diffchange-inline">interpreting cognitive model discovery algorithms across multiple data sets </del>in <del class="diffchange diffchange-inline">DataShop</del>. <del class="diffchange diffchange-inline"> </del></div></td><td class='diff-marker'>+</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>* <ins class="diffchange diffchange-inline">Shih, Scheines, & Koedinger will create a “Target Sequence Clustering” technique (Shih’s thesis) that will be applied to identify patterns in tutor log data that characterize good </ins>and <ins class="diffchange diffchange-inline">poor student learning strategies </ins>and <ins class="diffchange diffchange-inline">are predictive of individual differences </ins>in <ins class="diffchange diffchange-inline">student learning rate</ins>.  </div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>* <del class="diffchange diffchange-inline">Demonstrate </del>the <del class="diffchange diffchange-inline">use of </del>the <del class="diffchange diffchange-inline">model discovery infrastructure </del>for <del class="diffchange diffchange-inline">at least two discovery algorithms applied to at least 4 DataShop data sets</del>.  </div></td><td class='diff-marker'>+</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>* <ins class="diffchange diffchange-inline">Li, Cohen & Koedinger will continue their work to produce a model and demonstration of accelerated learning within </ins>the <ins class="diffchange diffchange-inline">SimStudent architecture.  We will extend past work that has demonstrated </ins>the <ins class="diffchange diffchange-inline">potential </ins>for <ins class="diffchange diffchange-inline">deep feature learning technique using probabilistic grammar learning, by integrating those machine learning techniques into SimStudent and testing whether SimStudent can learn algebra with only weak prior knowledge (shallow features) by acquiring deep features rather than being programmed with strong prior knowledge as was done in the past</ins>.  </div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>* <del class="diffchange diffchange-inline">For at least one of these data sets</del>, <del class="diffchange diffchange-inline">work with associated researchers </del>to <del class="diffchange diffchange-inline">perform a “close the loop” experiment whereby we test that </del>a <del class="diffchange diffchange-inline">better cognitive </del>model <del class="diffchange diffchange-inline">leads </del>to <del class="diffchange diffchange-inline">better or more efficient </del>student learning.   </div></td><td class='diff-marker'>+</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>* <ins class="diffchange diffchange-inline">With leveraged funding (DoE IES and NSF REESE), Matsuda, Booth</ins>, <ins class="diffchange diffchange-inline">& Koedinger will continue </ins>to <ins class="diffchange diffchange-inline">explore SimStudent as </ins>a model <ins class="diffchange diffchange-inline">algebra learning data in which differences in student prior knowledge (pre-requisite concepts) lead </ins>to <ins class="diffchange diffchange-inline">differences in </ins>student learning <ins class="diffchange diffchange-inline">rate.  The work of Li, Cohen, & Koedinger may contribute to this effort.</ins></div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>=== <del class="diffchange diffchange-inline">6th Month Milestone </del>===</div></td><td class='diff-marker'>+</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div> </div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del class="diffchange diffchange-inline">By March</del>, 2010 <del class="diffchange diffchange-inline">we will 1</del>) be <del class="diffchange diffchange-inline">able to run </del>the <del class="diffchange diffchange-inline">LFA algorithm </del>on <del class="diffchange diffchange-inline">PSLC data sets from the DataShop web services</del>, <del class="diffchange diffchange-inline">2</del>) <del class="diffchange diffchange-inline">have run model discovery with </del>using <del class="diffchange diffchange-inline">at least one algorithm on at least two </del>data <del class="diffchange diffchange-inline">sets</del>, <del class="diffchange diffchange-inline">and 3</del>) <del class="diffchange diffchange-inline">we will have designed and ideally run </del>the <del class="diffchange diffchange-inline">close</del>-the-<del class="diffchange diffchange-inline">loop experiment</del>.</div></td><td class='diff-marker'>+</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins class="diffchange diffchange-inline">=== Participants ===</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins class="diffchange diffchange-inline">Ken Koedinger & PhD students Ben Shih and Nan Li</ins>.  <ins class="diffchange diffchange-inline">Other contributors are Dr. William Cohen (Machine Learning; co-advisor of Nan Li), Dr. Richard Schienes (Philosophy, co-advisor of Ben Shih), Dr. Noboru Matsuda, Dr. Julie Booth, and the SimStudent and CTAT teams.</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div> </div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>=== <ins class="diffchange diffchange-inline">References </ins>===</div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins class="diffchange diffchange-inline">* Li, N., Cohen</ins>, <ins class="diffchange diffchange-inline">W. W., & Koedinger, K. R. (</ins>2010)<ins class="diffchange diffchange-inline">.  A computational model of accelerated future learning through feature recognition.  In Proceedings of the 10th International Conference of Intelligent Tutoring Systems.</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins class="diffchange diffchange-inline">* Matsuda, N., Cohen, W. W., Sewall, J., Lacerda, G., & Koedinger, K. R. (2008). Why tutored problem solving may </ins>be <ins class="diffchange diffchange-inline">better than example study: Theoretical implications from a simulated-student study. In B. P. Woolf, E. Aimeur, R. Nkambou & S. Lajoie (Eds.), Proceedings of </ins>the <ins class="diffchange diffchange-inline">International Conference </ins>on <ins class="diffchange diffchange-inline">Intelligent Tutoring Systems (pp. 111-121). Heidelberg, Berlin: Springer.</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins class="diffchange diffchange-inline">* Matsuda, N., Cohen, W. W., Sewall, J., Lacerda, G., & Koedinger</ins>, <ins class="diffchange diffchange-inline">K. R. (2007</ins>)<ins class="diffchange diffchange-inline">. Evaluating a simulated student </ins>using <ins class="diffchange diffchange-inline">real students </ins>data <ins class="diffchange diffchange-inline">for training and testing. In C. Conati</ins>, <ins class="diffchange diffchange-inline">K. McCoy & G. Paliouras (Eds.</ins>)<ins class="diffchange diffchange-inline">, Proceedings of </ins>the <ins class="diffchange diffchange-inline">international conference on User Modeling (LNAI 4511) (pp. 107</ins>-<ins class="diffchange diffchange-inline">116). Berlin, Heidelberg: Springer.</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins class="diffchange diffchange-inline">* Matsuda, N., Lee, A., Cohen, W. W., & Koedinger, K. R. (2009). A computational model of how learner errors arise from weak prior knowledge. In Proceedings of the Conference of </ins>the <ins class="diffchange diffchange-inline">Cognitive Science Society.</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins class="diffchange diffchange-inline">* Shih, B., Koedinger, K. R., & Scheines, R. (2008). A response time model for bottom</ins>-<ins class="diffchange diffchange-inline">out hints as worked examples. In Proceedings of the 1st International Conference on Educational Data Mining</ins>.</div></td></tr>
</table>Koedingerhttps://learnlab.org/wiki/index.php?title=Koedinger_-_Toward_a_model_of_accelerated_future_learning&diff=11055&oldid=prevKoedinger at 20:10, 13 September 20102010-09-13T20:10:05Z<p></p>
<table class="diff diff-contentalign-left" data-mw="interface">
<col class="diff-marker" />
<col class="diff-content" />
<col class="diff-marker" />
<col class="diff-content" />
<tr class="diff-title" lang="en">
<td colspan="2" style="background-color: #fff; color: #222; text-align: center;">← Older revision</td>
<td colspan="2" style="background-color: #fff; color: #222; text-align: center;">Revision as of 20:10, 13 September 2010</td>
</tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l8" >Line 8:</td>
<td colspan="2" class="diff-lineno">Line 8:</td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>3. For at least one of these data sets, work with associated researchers to perform a “close the loop” experiment whereby we test whether a better cognitive model leads to better or more efficient student learning.  </div></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>3. For at least one of these data sets, work with associated researchers to perform a “close the loop” experiment whereby we test whether a better cognitive model leads to better or more efficient student learning.  </div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'>−</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>== Integrated Research Results <del class="diffchange diffchange-inline">and High Profile Publication </del>==</div></td><td class='diff-marker'>+</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>== Integrated Research Results ==</div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>Establishing that cognitive models of academic domain knowledge in math, science, and language can be discovered from data would be an important scientific achievement.  The achievement will be greater to the extent that the discovered models involve deep or integrative knowledge components not directly apparent in surface task structure (e.g., model discovery in the Geometry area domain isolated a problem decomposition skill).  The statistical model structure of competing discovery algorithms promises to shed new light on the nature or extent of regularities or laws of learning, like the power or exponential shape of learning curves, whether the complexity of task behavior is due to human or domain characteristics (the ant on the beach question), whether or not there are systematic individual differences in student learning rates.  <del class="diffchange diffchange-inline">We expect integrative results of this project can be published in high-profile general journals (e.g., Science or Nature) or more specific technical (e.g., Machine Learning or JMLR) or psychological journals (e.g., Cognitive Science or Learning Science). </del></div></td><td class='diff-marker'>+</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>Establishing that cognitive models of academic domain knowledge in math, science, and language can be discovered from data would be an important scientific achievement.  The achievement will be greater to the extent that the discovered models involve deep or integrative knowledge components not directly apparent in surface task structure (e.g., model discovery in the Geometry area domain isolated a problem decomposition skill).  The statistical model structure of competing discovery algorithms promises to shed new light on the nature or extent of regularities or laws of learning, like the power or exponential shape of learning curves, whether the complexity of task behavior is due to human or domain characteristics (the ant on the beach question), whether or not there are systematic individual differences in student learning rates.   </div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>== Year 6 Project Deliverables ==</div></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>== Year 6 Project Deliverables ==</div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del class="diffchange diffchange-inline"> . </del>Develop code and human-computer interfaces for applying, comparing and interpreting cognitive model discovery algorithms across multiple data sets in DataShop.   </div></td><td class='diff-marker'>+</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins class="diffchange diffchange-inline">* </ins>Develop code and human-computer interfaces for applying, comparing and interpreting cognitive model discovery algorithms across multiple data sets in DataShop.   </div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del class="diffchange diffchange-inline"> . </del>Demonstrate the use of the model discovery infrastructure for at least two discovery algorithms applied to at least 4 DataShop data sets.  </div></td><td class='diff-marker'>+</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins class="diffchange diffchange-inline">* </ins>Demonstrate the use of the model discovery infrastructure for at least two discovery algorithms applied to at least 4 DataShop data sets.  </div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del class="diffchange diffchange-inline"> . </del>For at least one of these data sets, work with associated researchers to perform a “close the loop” experiment whereby we test that a better cognitive model leads to better or more efficient student learning.   </div></td><td class='diff-marker'>+</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins class="diffchange diffchange-inline">* </ins>For at least one of these data sets, work with associated researchers to perform a “close the loop” experiment whereby we test that a better cognitive model leads to better or more efficient student learning.   </div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>6th Month Milestone</div></td><td class='diff-marker'>+</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins class="diffchange diffchange-inline">=== </ins>6th Month Milestone <ins class="diffchange diffchange-inline">===</ins></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>By March, 2010 we will 1) be able to run the LFA algorithm on PSLC data sets from the DataShop web services, 2) have run model discovery with using at least one algorithm on at least two data sets, and 3) we will have designed and ideally run the close-the-loop experiment.</div></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>By March, 2010 we will 1) be able to run the LFA algorithm on PSLC data sets from the DataShop web services, 2) have run model discovery with using at least one algorithm on at least two data sets, and 3) we will have designed and ideally run the close-the-loop experiment.</div></td></tr>
</table>Koedingerhttps://learnlab.org/wiki/index.php?title=Koedinger_-_Toward_a_model_of_accelerated_future_learning&diff=10393&oldid=prevKoedinger: New page: == Project Overview == This project will address goal 1 of the CMDM thrust and in particular use DataShop datasets (90 in 5 years) to produce better cognitive models and verify the models ...2009-12-17T19:01:23Z<p>New page: == Project Overview == This project will address goal 1 of the CMDM thrust and in particular use DataShop datasets (90 in 5 years) to produce better cognitive models and verify the models ...</p>
<p><b>New page</b></p><div>== Project Overview ==<br />
This project will address goal 1 of the CMDM thrust and in particular use DataShop datasets (90 in 5 years) to produce better cognitive models and verify the models with in vivo experiments. Cognitive models drive the great many instructional decisions that automated tutoring currently make, whether it is how to organize instructional messages, sequence topics and problems in a curriculum, adapt pacing to student needs, or select appropriate materials and tasks to adapt to student needs. Cognitive models also appear critical to accurate assessment of self-regulated learning skills or motivational states.<br />
Multiple algorithms have been developed for automated discovery of the attributes or factors that make up a cognitive model (or a "Q matrix") including various Q-matrix discovery algorithms like Rule Spaces, Knowledge Spaces, Learning Factors Analysis (LFA), and Bayesian exponential-family PCA. This project will create an infrastructure for automatically applying such algorithms to data sets in the DataShop, discovering better cognitive models, and evaluating whether such models improve tutors.<br />
<br />
== Planned accomplishments for PSLC Year 6 ==<br />
1. Develop code and human-computer interfaces for applying, comparing and interpreting cognitive model discovery algorithms across multiple data sets in DataShop. We will document processes for how the algorithms, like LFA, combine automation and human input to discover or improve cognitive models of specific learning domains. <br />
2. Demonstrate the use of the model discovery infrastructure (#1) for at least two discovery algorithms applied to at least 4 DataShop data sets. We will target at least one math (Geometry area and/or Algebra equation solving), one science (Physics kinematics), and one language (English articles) domain. <br />
3. For at least one of these data sets, work with associated researchers to perform a “close the loop” experiment whereby we test whether a better cognitive model leads to better or more efficient student learning. <br />
<br />
== Integrated Research Results and High Profile Publication ==<br />
Establishing that cognitive models of academic domain knowledge in math, science, and language can be discovered from data would be an important scientific achievement. The achievement will be greater to the extent that the discovered models involve deep or integrative knowledge components not directly apparent in surface task structure (e.g., model discovery in the Geometry area domain isolated a problem decomposition skill). The statistical model structure of competing discovery algorithms promises to shed new light on the nature or extent of regularities or laws of learning, like the power or exponential shape of learning curves, whether the complexity of task behavior is due to human or domain characteristics (the ant on the beach question), whether or not there are systematic individual differences in student learning rates. We expect integrative results of this project can be published in high-profile general journals (e.g., Science or Nature) or more specific technical (e.g., Machine Learning or JMLR) or psychological journals (e.g., Cognitive Science or Learning Science). <br />
<br />
== Year 6 Project Deliverables ==<br />
. Develop code and human-computer interfaces for applying, comparing and interpreting cognitive model discovery algorithms across multiple data sets in DataShop. <br />
. Demonstrate the use of the model discovery infrastructure for at least two discovery algorithms applied to at least 4 DataShop data sets. <br />
. For at least one of these data sets, work with associated researchers to perform a “close the loop” experiment whereby we test that a better cognitive model leads to better or more efficient student learning. <br />
6th Month Milestone<br />
By March, 2010 we will 1) be able to run the LFA algorithm on PSLC data sets from the DataShop web services, 2) have run model discovery with using at least one algorithm on at least two data sets, and 3) we will have designed and ideally run the close-the-loop experiment.</div>Koedinger