Difference between revisions of "Collected User Requests"

From LearnLab
Jump to: navigation, search
(Non-KC Modeling)
(Upload model and apply it to new data set<br>)
Line 44: Line 44:
 
** Code to display GUI to choose which data sets to use, calls model code, re-import to DataShop
 
** Code to display GUI to choose which data sets to use, calls model code, re-import to DataShop
 
** Good to have a way to apply many models, as soon as you import a data set
 
** Good to have a way to apply many models, as soon as you import a data set
* Phil has an idea that maybe fits within this one. Please move if there's a better category [bl]
+
* Phil has an idea that maybe fits within this one. Please move if there's a better category [Brett Leber]
 
<blockquote>This [''transaction? kc? --ed.''] relabeling is really mostly about enabling modeling in DataShop right? With this in mind, I think that it is actually a higher priority to have model alternatives in DataShop.... E.g. Investigators should be able to give you chunks of Java code according to a certain specification, and DataShop should be then able to run these over datasets (perhaps after a certain series of QA occurs according to an SOP) when the investigator clicks some button in DataShop.... Obviously this is a much large project than adding columns, but it is also much more important in my mind. <br>''--Phil Pavlik, email to Brett on 1/14/2009''</blockquote>  
 
<blockquote>This [''transaction? kc? --ed.''] relabeling is really mostly about enabling modeling in DataShop right? With this in mind, I think that it is actually a higher priority to have model alternatives in DataShop.... E.g. Investigators should be able to give you chunks of Java code according to a certain specification, and DataShop should be then able to run these over datasets (perhaps after a certain series of QA occurs according to an SOP) when the investigator clicks some button in DataShop.... Obviously this is a much large project than adding columns, but it is also much more important in my mind. <br>''--Phil Pavlik, email to Brett on 1/14/2009''</blockquote>  
 
* Examples:  
 
* Examples:  

Revision as of 15:27, 2 September 2009

Annotations

Have a link from the DataShop to the Theory Wiki (Dataset to Project Page)

  • Michael Bett, ET Mtg 11/14/2007
  • From meeting minutes:
    • Michael: Link the dataset to the project page? In the pipeline have a clickable link to the project page (make project name clickable).
    • Brett: Link to a dataset directly? Is that obvious to users? Click on dataset link -> log in -> redirected back to dataset.

Annotations on transaction level

  • Ryan, ET Mtg 12/5/2007
  • Has models which can annotate something like: gaming, bored, etc.

Annotations on the student level

  • Ido Roll, Interview with Brett Leber, 1/19/2009
  • Can annotate on student level, a percentage the student is gaming
    • Ido has mentioned annotations on the student level

Dataset Discussion - Capture data-integrity issues

  • Ken Koedinger, Team Meeting, 8/15/2009
  • As a stakeholder in the DataShop project, I want to capture and publicize the data-integrity issues discovered with data sets so that data is better documented (and so we've fulfilled a promise to our funders to better document data).
  • As a user of DataShop, I want to discuss datasets and have that discussion attached to the dataset so that others can comment and better understand any data-integrity issues I've found.

Linking to internal pages

Data Modeling

Non-KC Modeling

Automatic Distillation

  • Ryan, Summer 2008, Startup Memo
  • As an educational data miner wishing to develop a machine learned model with PSLC data, I would like to be able to automatically distill data features (e.g. custom fields) commonly used in past educational data mining research for a new data set (see, for instance, Baker, Corbett, Roll, & Koedinger, 2008 in UMUAI)
  • Could be implemented as a plug-in
  • Also interested in this feature idea:
    • Dan Franklin, Oct 2008

Upload model and apply it to new data set

  • EDM researcher would like to take a model, expressable as a linear formula on DataShop fields, or a simple code procedure (e.g. Bayesian Knowledge Tracing, which Ryan has code for), and apply it to a new data set [Maxine, Sept 2008; Ryan, Sept 2008; required for prior Hao request]
  • May work best as a plug-in
    • Code to display GUI to choose which data sets to use, calls model code, re-import to DataShop
    • Good to have a way to apply many models, as soon as you import a data set
  • Phil has an idea that maybe fits within this one. Please move if there's a better category [Brett Leber]

This [transaction? kc? --ed.] relabeling is really mostly about enabling modeling in DataShop right? With this in mind, I think that it is actually a higher priority to have model alternatives in DataShop.... E.g. Investigators should be able to give you chunks of Java code according to a certain specification, and DataShop should be then able to run these over datasets (perhaps after a certain series of QA occurs according to an SOP) when the investigator clicks some button in DataShop.... Obviously this is a much large project than adding columns, but it is also much more important in my mind.
--Phil Pavlik, email to Brett on 1/14/2009

  • Examples:
    • Example: running gaming detector in multiple tutors and comparing gaming frequencies
    • Example: applying Bayesian Knowledge Tracing to a new data set from the same LearnLab
    • Example: applying Ben Shih's models to many data sets [Ben Shih should be included in design of this feature; he is interested, and has a lot of good ideas]

KC Modeling

Automatically discovering new KC model

  • Vincent Aleven, Sept 2008
  • Possible to run some code (perhaps Hao's KC model selection code, perhaps something else generated by CMDM thrust) to find new best KC model.
  • As a learning sciences researcher, I would like DataShop to discover a new/better KC model for me.
  • Could be done as a plug-in

Help

Home Page

Import

Miscellaneous

Analyses by LearnLab

  • Organize data by LearnLab, not by "data set" [Ryan, Aug 2008; Bob, Sep 2008; Maxine, Sep 2008]
  • Essentially, current data sets become samples, but the top-level unit is the LearnLab. You can take every data set in a LearnLab together as a sample.
  • Implies being able to run analyses across data sets, and export multiple data sets together; to create multi-data set samples
  • As a user of DataShop, I would like to look at learning curves for all Algebra data together (for example), or export all Algebra data
  • Important long-term, but is a lot of work -- in particular, we need to solve scalability issues first

Navigation Bar

New Visualizations/Reports

Reports

Dataset Info

Error Report

Export

Learning Curve

Performance Profiler

Sample Selector

Web Services