Collected User Requests

Annotations

Have a link from the DataShop to the Theory Wiki (Dataset to Project Page)

• Michael Bett, ET Mtg 11/14/2007
• From meeting minutes:
  • Michael: Link the dataset to the project page? In the pipeline have a clickable link to the project page (make project name clickable).
  • Brett: Link to a dataset directly? Is that obvious to users? Click on dataset link -> log in -> redirected back to dataset.

Annotations on transaction level

• Ryan, ET Mtg 12/5/2007
• Has models which can annotate something like: gaming, bored, etc.

Annotations on the student level

• Ido Roll, Interview with Brett Leber, 1/19/2009
• Can annotate on student level, a percentage the student is gaming
  • Ido has mentioned annotations on the student level

Dataset Discussion - Capture data-integrity issues

• Ken Koedinger, Team Meeting, 8/15/2009
• As a stakeholder in the DataShop project, I want to capture and publicize the data-integrity issues discovered with data sets so that data is better documented (and so we've fulfilled a promise to our funders to better document data).
• As a user of DataShop, I want to discuss datasets and have that discussion attached to the dataset so that others can comment and better understand any data-integrity issues I've found.

Linking to internal pages

• Alida, 10/18/2007
• It would be handy if a link can be saved to any dataset, sample, page in the DataShop.
• Currently, https://pslc-qa.andrew.cmu.edu/datashop/DatasetInfo?datasetId=793 works if you are already logged in.

Data Modeling

Non-KC Modeling

Automatic Distillation

• Ryan, Summer 2008, Startup Memo
• As an educational data miner wishing to develop a machine learned model with PSLC data, I would like to be able to automatically distill data features (e.g. custom fields) commonly used in past educational data mining research for a new data set (see, for instance, Baker, Corbett, Roll, & Koedinger, 2008 in UMUAI)
• Could be implemented as a plug-in
• Also interested in this feature idea:
  • Dan Franklin, Oct 2008

KC Modeling

Automatically discovering new KC model

• Vincent Aleven, Sept 2008
• Possible to run some code (perhaps Hao's KC model selection code, perhaps something else generated by CMDM thrust) to find new best KC model.
• As a learning sciences researcher, I would like DataShop to discover a new/better KC model for me.
• Could be done as a plug-in

Miscellaneous

Analyses by LearnLab

• Organize data by LearnLab, not by "data set" [Ryan, Aug 2008; Bob, Sep 2008; Maxine, Sep 2008]
• Essentially, current data sets become samples, but the top-level unit is the LearnLab. You can take every data set in a LearnLab together as a sample.
• Implies being able to run analyses across data sets, and export multiple data sets together; to create multi-data set samples
• As a user of DataShop, I would like to look at learning curves for all Algebra data together (for example), or export all Algebra data
• Important long-term, but is a lot of work -- in particular, we need to solve scalability issues first