Gordon - Temporal learning for EDM

Participants

Faculty: Geoff Gordon; Students: Ajit Singh, Byron Boots, Nick Armstrong-Crews

Project Overview

To this point, our efforts to apply general purpose machine learning tools to educational data mining have focused chiefly on static data: for example, given a student-item matrix of scores, learn latent factors which predict the performance of a new student on previously-known items. To accurately model the processes of learning and problem solving, we need instead to look at temporal data.

Models such as the Additive Factor Model begin to address temporal data by computing additional features such as counts of opportunities to apply a skill, but we hope to go further and automatically identify additional temporally-changing features. In more detail, we propose to apply general-purpose time-series learning tools to data from students interacting with a tutor, to attempt to predict domain-specific performance over time. We will apply these tools to learn domain-specific predictive models and test these models on DataShop data.

The resulting models should eventually provide input to tools which design new tutor interventions and test whether they improve learning, such as those developed in Chi - Induction of Adaptive Pedagogical Tutorial Tactics. For this synergy to work, we need to make sure that our learned models are (a) able to predict the effects of intervention, and (b) in a form compatible with our chosen planning algorithms (such as point-based planning, discussed more in the linked project).

We identify two separate types of temporal data. First, during the course of a single problem, a student can interact with the tutor multiple times in a flexible order. Early interactions can serve to predict later ones: for example, if a student becomes confused early in the problem, all future interactions may be affected until we resolve the student’s confusion. Or, in problems where multiple solution strategies may be effective, we may be able to determine early on which strategy a student is following, and use that knowledge to predict future actions.

Second, a student can interact with the tutor multiple times across multiple problems. In this case, our goal will be to identify a student’s progress acquiring the different skills or knowledge components required to solve the problems.

In either case, the goal of our time-series modeling techniques will be to discover the latent factors which influence the observed performance data. In the within-problem case, these latent factors might include the overall solution strategy that the student is attempting to apply, the current step within this solution strategy, or the different types of confusion the student could exhibit. In the across-problems case, these latent factors would include the student’s level of proficiency at each relevant skill or knowledge component. We can even have both types of temporal data at once: a student can interact multiple times on each problem, and solve multiple problems over time. In this case, we can simultaneously seek to discover both kinds of latent factors.

From a technical point of view, we expect to use some or all of the following tools:

• Autoregression, using earlier responses directly as inputs to a linear or nonlinear regression to predict later responses
• Factor analysis, decomposing a matrix of measured data X into a function of low-rank factors: X ≈ f (U V)
• Subspace identification, combining autoregression with factor analysis to identify latent factors related to variation over time

We will use existing versions of the above tools, and also develop extensions motivated by the characteristics of educational data. If the current project works out well, a possible follow-on project would be to combine data sets from multiple domains. By controlling for the domain-specific factors learned for each individual domain, we could attempt to uncover domain-general learning principles.