Theory Wiki - User contributions [en]

Dialogue Message Format

2010-02-26T14:54:34Z

Jstamper: New page: '''Status: Requirements Documents Needed''' == User Story == As a member of the Social and Communicative Factors thrust, I want to be able to store and report on dialogue data in the for...

'''Status: Requirements Documents Needed'''

== User Story ==

As a member of the Social and Communicative Factors thrust, I want to be able to store and report on dialogue data in the form of transcripts, possibly including the actual audio or video data from which the transcripts came from.

== Notes/Comments ==

* I am envisioning a separate set of tables for this type of data and a message format similar to the tutor message format, but satisfying the unique nature and reporting needs of dialogue data. -- jstamper 02/26/2010

 
----
See complete [[DataShop Feature Wish List]].
[[Category:Protected]]
[[Category:DataShop]]

DataShop Feature Wish List

2010-02-26T14:46:56Z

Jstamper: /* Prioritized Features */

Below are two lists of features. The features that we have prioritized and decided to implement are in the first, ordered list. The features that the DataShop team and community are discussing are in an unordered list on the page [[Collected User Requests]]. Click on a feature to get more information about it, such as a description, rationale for building it, and its status.

'''You can help!'''

If you think a feature is important, vote for it by putting your name to the right of the feature. Discuss the feature on the comments section of that feature's page. We'll use these votes and the dialogue that develops to prioritize features.

Don't see a feature on the prioritized list? There's a good chance it's on the '''[[Collected User Requests]]''' page. You can add feature ideas there and discuss the existing ones. Include your comment, name, and date to vote on feature ideas there.

'''Tip:''' Easily sign your username and the current date/time by inserting four tildes (<nowiki>~~~~</nowiki>); insert just your username with three tildes.

''See features we are building now, [[DataShop 4.x Features]], and ones we have built, [[DataShop 3.x Features]].''

== Prioritized Features ==

# [[Web Services]] (add custom fields to transactions) — Vote: Ryan Baker(1), John Stamper(1)
# [[Adding Custom Fields through Web Application]] -- Vote: Ryan Baker (2)
# [[Error Bars]] — Vote: Ken Koedinger(1)
# [[Metrics]] — Vote: John Stamper (2), Ryan Baker (3)
# [[Push Button Import]] — Carnegie Learning, John Stamper
# [[KC Model in Transaction Export]] — Vote: Vincent Aleven(2)
# [[Student Filter Dialog]]
# [[Milliseconds]]
# [[LFA on Sample]] — Vote: Ken Koedinger(3)
# [[Place for General Papers]]
# [[Performance Metrics]]
# [[KC Model Sort]]
# [[Ability to display step-custom-fields in graphs]]
# [[Scalability]] -- Vote: Ryan Baker (4)
# [[Terms of Use]] — Steve Ritter
# [[Dialogue Message Format]]

== Unordered Features ==
We have a long list of feature requests that have not been prioritized. Please see the
'''[[Collected User Requests]]'''.

== How to Request a Feature ==
* [[Write a User Story]]
* [[Create a Feature Page]]
* Add Link to Feature on [[Collected User Requests]] page.
 
----
See completed [[DataShop 3.x Features]] 
See on-going [[DataShop 4.x Features]] 
See unordered [[Collected User Requests]]
[[Category:Protected]]
[[Category:DataShop]]

Koedinger - Discovery of Domain-Specific Cognitive Models

2009-11-20T22:42:18Z

Jstamper:

Ken Koedinger and John Stamper

== Project Overview ==
This project will address goal 1 of the CMDM thrust and in particular use DataShop datasets (90 in 5 years) to produce better cognitive models and verify the models with in vivo experiments. Cognitive models drive the great many instructional decisions that automated tutoring currently make, whether it is how to organize instructional messages, sequence topics and problems in a curriculum, adapt pacing to student needs, or select appropriate materials and tasks to adapt to student needs. Cognitive models also appear critical to accurate assessment of self-regulated learning skills or motivational states.
Multiple algorithms have been developed for automated discovery of the attributes or factors that make up a cognitive model (or a "Q matrix") including various Q-matrix discovery algorithms like Rule Space, Knowledge Spaces, Learning Factors Analysis (LFA), exponential-family PCA. This project will create an infrastructure for automatically applying such algorithms to data sets in the DataShop, discovering better cognitive models, and evaluating whether such models improve tutors.

== Planned accomplishments for PSLC Year 6 (Oct 09 to Oct 10) ==
1. Develop code and human-computer interfaces for applying, comparing and interpreting cognitive model discovery algorithms across multiple data sets in DataShop. We will document processes for how the algorithms, like LFA, combine automation and human input to discover or improve cognitive models of specific learning domains.

2. Demonstrate the use of the model discovery infrastructure (#1) for at least two discovery algorithms applied to at least 4 DataShop data sets. We will target at least one math (Geometry area and/or Algebra equation solving), one science (Physics kinematics), and one language (English articles) domain.

3. For at least one of this data sets, work with associated researchers to perform a "close the loop" experiment whereby we demonstrate that a better cognitive model leads to better or more efficient student learning.

== Integrated Research Results and High Profile Publication ==
Establishing that cognitive models of academic domain knowledge in math, science, and language can be discovered from data would be an important scientific achievement. The achievement will be greater to the extent that the discovered models involve deep or integrative knowledge components not directly apparent in surface task structure (e.g., model discovery in the Geometry area domain isolated a problem decomposition skill). The statistical model structure of competing discovery algorithms promises to shed new light on the nature or extent of regularities or laws of learning, like the power or exponential shape of learning curves, whether the complexity of task behavior is due to human or domain characteristics (the ant on the beach question), whether or not there are systematic individual differences in student learning rates. We expect integrative results of this project can be published in high-profile general journals (Science or Nature) or more specific technical (e.g., Machine Learning) or psychological journals (e.g., Cognitive Science or Learning Science).

Koedinger - Discovery of Domain-Specific Cognitive Models

2009-11-20T22:41:58Z

Jstamper:

== Discovery of Domain-Specific Cognitive Models and Design of Better Tutors ==
Ken Koedinger and John Stamper

== Project Overview ==
This project will address goal 1 of the CMDM thrust and in particular use DataShop datasets (90 in 5 years) to produce better cognitive models and verify the models with in vivo experiments. Cognitive models drive the great many instructional decisions that automated tutoring currently make, whether it is how to organize instructional messages, sequence topics and problems in a curriculum, adapt pacing to student needs, or select appropriate materials and tasks to adapt to student needs. Cognitive models also appear critical to accurate assessment of self-regulated learning skills or motivational states.
Multiple algorithms have been developed for automated discovery of the attributes or factors that make up a cognitive model (or a "Q matrix") including various Q-matrix discovery algorithms like Rule Space, Knowledge Spaces, Learning Factors Analysis (LFA), exponential-family PCA. This project will create an infrastructure for automatically applying such algorithms to data sets in the DataShop, discovering better cognitive models, and evaluating whether such models improve tutors.

== Planned accomplishments for PSLC Year 6 (Oct 09 to Oct 10) ==
1. Develop code and human-computer interfaces for applying, comparing and interpreting cognitive model discovery algorithms across multiple data sets in DataShop. We will document processes for how the algorithms, like LFA, combine automation and human input to discover or improve cognitive models of specific learning domains.

2. Demonstrate the use of the model discovery infrastructure (#1) for at least two discovery algorithms applied to at least 4 DataShop data sets. We will target at least one math (Geometry area and/or Algebra equation solving), one science (Physics kinematics), and one language (English articles) domain.

3. For at least one of this data sets, work with associated researchers to perform a "close the loop" experiment whereby we demonstrate that a better cognitive model leads to better or more efficient student learning.

== Integrated Research Results and High Profile Publication ==
Establishing that cognitive models of academic domain knowledge in math, science, and language can be discovered from data would be an important scientific achievement. The achievement will be greater to the extent that the discovered models involve deep or integrative knowledge components not directly apparent in surface task structure (e.g., model discovery in the Geometry area domain isolated a problem decomposition skill). The statistical model structure of competing discovery algorithms promises to shed new light on the nature or extent of regularities or laws of learning, like the power or exponential shape of learning curves, whether the complexity of task behavior is due to human or domain characteristics (the ant on the beach question), whether or not there are systematic individual differences in student learning rates. We expect integrative results of this project can be published in high-profile general journals (Science or Nature) or more specific technical (e.g., Machine Learning) or psychological journals (e.g., Cognitive Science or Learning Science).

Koedinger - Discovery of Domain-Specific Cognitive Models

2009-11-20T22:38:54Z

Jstamper: New page: Discovery of Domain-Specific Cognitive Models and Design of Better Tutors Ken Koedinger and John Stamper Project Overview This project will address goal 1 of the CMDM thrust and in parti...

Discovery of Domain-Specific Cognitive Models and Design of Better Tutors
Ken Koedinger and John Stamper

Project Overview
This project will address goal 1 of the CMDM thrust and in particular use DataShop datasets (90 in 5 years) to produce better cognitive models and verify the models with in vivo experiments. Cognitive models drive the great many instructional decisions that automated tutoring currently make, whether it is how to organize instructional messages, sequence topics and problems in a curriculum, adapt pacing to student needs, or select appropriate materials and tasks to adapt to student needs. Cognitive models also appear critical to accurate assessment of self-regulated learning skills or motivational states.
Multiple algorithms have been developed for automated discovery of the attributes or factors that make up a cognitive model (or a "Q matrix") including various Q-matrix discovery algorithms like Rule Space, Knowledge Spaces, Learning Factors Analysis (LFA), exponential-family PCA. This project will create an infrastructure for automatically applying such algorithms to data sets in the DataShop, discovering better cognitive models, and evaluating whether such models improve tutors.

Planned accomplishments for PSLC Year 6 (Oct 09 to Oct 10)
1. Develop code and human-computer interfaces for applying, comparing and interpreting cognitive model discovery algorithms across multiple data sets in DataShop. We will document processes for how the algorithms, like LFA, combine automation and human input to discover or improve cognitive models of specific learning domains.
2. Demonstrate the use of the model discovery infrastructure (#1) for at least two discovery algorithms applied to at least 4 DataShop data sets. We will target at least one math (Geometry area and/or Algebra equation solving), one science (Physics kinematics), and one language (English articles) domain.
3. For at least one of this data sets, work with associated researchers to perform a "close the loop" experiment whereby we demonstrate that a better cognitive model leads to better or more efficient student learning.

Integrated Research Results and High Profile Publication
Establishing that cognitive models of academic domain knowledge in math, science, and language can be discovered from data would be an important scientific achievement. The achievement will be greater to the extent that the discovered models involve deep or integrative knowledge components not directly apparent in surface task structure (e.g., model discovery in the Geometry area domain isolated a problem decomposition skill). The statistical model structure of competing discovery algorithms promises to shed new light on the nature or extent of regularities or laws of learning, like the power or exponential shape of learning curves, whether the complexity of task behavior is due to human or domain characteristics (the ant on the beach question), whether or not there are systematic individual differences in student learning rates. We expect integrative results of this project can be published in high-profile general journals (Science or Nature) or more specific technical (e.g., Machine Learning) or psychological journals (e.g., Cognitive Science or Learning Science).

Metrics

2009-09-25T14:45:52Z

Jstamper:

'''Status: Prioritization Needed'''

== User Story ==

As a administrator of DataShop, I want to see performance metrics that are updated at a regular interval so that I can quickly identify and respond to bottlenecks in the DataShop as an administrator, and also to make sure that we are seeing growth and improvement in DataShop scalability.

== Notes/Comments ==

* Steps to complete
* Create a list of metrics.
* Determine where the metrics will be viewable
* Determine who can see which metrics
* Write procedures to calculate metrics
* Integrate with application

 
----
See complete [[DataShop Feature Wish List]].
[[Category:Protected]]
[[Category:DataShop]]

Metrics

2009-09-25T14:39:26Z

Jstamper: New page: test

test

DataShop Feature Wish List

2009-09-25T14:37:29Z

Jstamper:

Below are two lists of features. The features that we have prioritized and decided to implement are in the first, ordered list. The features that the DataShop team and community are discussing are in an unordered list on the page [[Collected User Requests]]. Click on a feature to get more information about it, such as a description, rationale for building it, and its status.

'''You can help!'''

If you think a feature is important, vote for it by putting your name to the right of the feature. Discuss the feature on the comments section of that feature's page. We'll use these votes and the dialogue that develops to prioritize features.

Don't see a feature on the prioritized list? There's a good chance it's on the '''[[Collected User Requests]]''' page. You can add feature ideas there and discuss the existing ones. Include your comment, name, and date to vote on feature ideas there.

'''Tip:''' Easily sign your username and the current date/time by inserting four tildes (<nowiki>~~~~</nowiki>); insert just your username with three tildes.

''See features we are building now, [[DataShop 4.x Features]], and ones we have built, [[DataShop 3.x Features]].''

== Prioritized Features ==

# [[Web Services]] (authentication, get datasets, get samples)
# [[Are We Logging]] — Vote: Vincent Aleven(1)
# [[Project Announcements in DataShop Web Application]]
# [[Learning Curve Point Info Details: Add Frequency]] — Vote: Ken Koedinger
# [[Web Services]] (get transactions)
# [[Web Services]] (add custom fields to transactions) — Vote: Ryan Baker(1)
# [[Web Services]] (get student-step rollup)
# [[Adding Custom Fields through Web Application]]
# [[XML Extractor]] - Vote: Steve Ritter
# [[Error Bars]] - Vote: Ken Koedinger(1)
# [[Condition in Student-Step Rollup]] — Vote: Ken Koedinger(2), Vincent Aleven(3)
# [[Import File Request]]
# [[KC Model in Transaction Export]] — Vote: Vincent Aleven(2)
# [[LFA on Sample]] — Vote: Ken Koedinger(3)
# [[Student Filter Dialog]]
# [[Milliseconds]]
# [[DS680]]
# [[Push Button Import]]
# [[Place for General Papers]]
# [[KC Model Sort]]
# [[Ability to display step-custom-fields in graphs]]
# [[Scalability]]
# [[Metrics]] — Vote: John Stamper (1)

== Unordered Features ==
We have a long list of feature requests that have not been prioritized. Please see the
'''[[Collected User Requests]]'''.

== How to Request a Feature ==
* [[Write a User Story]]
* [[Create a Feature Page]]
* Add Link to Feature on [[Collected User Requests]] page.

[[Category:Protected]]
[[Category:DataShop]]

Create a Feature Page

2009-09-16T14:51:41Z

Jstamper: Undo revision 9913 by Jstamper (Talk)

'''Status: Prioritization Needed'''

== User Story ==

As a [type of user], I want to [perform some task] so that I can [achieve some goal].

== Notes/Comments ==

* Describe your problem and what you need here. [name date]

 
----
See complete [[DataShop Feature Wish List]].
[[Category:Protected]]
[[Category:DataShop]]

Create a Feature Page

2009-09-16T14:50:58Z

Jstamper: Undo revision 9914 by Jstamper (Talk)

'''Status: Prioritization Needed'''

== User Story ==

As a administrative user, I want to know how much data is in the DataShop so that I can give answer the question "how much data is in the DataShop".

== Notes/Comments ==

* Describe your problem and what you need here. [name date]

 
----
See complete [[DataShop Feature Wish List]].
[[Category:Protected]]
[[Category:DataShop]]

Create a Feature Page

2009-09-16T14:48:19Z

Jstamper: /* Notes/Comments */

'''Status: Prioritization Needed'''

== User Story ==

As a administrative user, I want to know how much data is in the DataShop so that I can give answer the question "how much data is in the DataShop".

== Notes/Comments ==

* Useful metrics include:
# of projects
# of datasets
# of software-student transactions
# of student hours available for analysis [John 9/16/2009]

 
----
See complete [[DataShop Feature Wish List]].
[[Category:Protected]]
[[Category:DataShop]]

Create a Feature Page

2009-09-16T14:46:12Z

Jstamper: /* User Story */