Pavlik - Difficulty and Strategy

Background

Currently the FaCT (Fact and Concept Training System) is deployed in the Chinese Learnlab and has been used in French and ESL Learnlabs for prior work. The FaCT system allows both randomized and adaptive model based scheduling of single step practice items either in the lab or integrated into a class. In past in vivo studies, some students do not seem to be motivated to complete more than the minimum work with the adaptive system to learn vocabulary, in contrast to lab studies that show that the system was is effective and might therefore be expected to lead to higher persistence in students. There are several motivational and metacognitive factors that could be responsible for the relatively low usage and/or poor performance of some students.

• Students find the tutor to be too difficult or too easy
• Students do not perceive a value to the practice
• Students do not expect to do well
• Students do not use appropriate strategies in the tutor

We plan to conduct in-vivo experiments to determine how interventions to modify these factors may result in improvements in both immediate learning, acceleration of future learning and longer-term transfer performance to in class measures.

A previous pilot study (with about 80 students participating) looked at the relationship between various aspects of performance and aspects of self-regulation, strategy use, and motivation, as measured by Pintrich’s Motivated Strategies for Learning Questionnaire (MSLQ; (Pintrich & de Groot, 1990). Two months after taking the MSLQ students were assigned the FaCT system vocabulary practice for Lesson 17 of their curriculum for 30 minutes. Two weeks following this practice there was a post practice survey that asked questions about perceived difficulty, self-regulation and strategies, and perceived value. While the results have not been fully analyzed, in Experiment 1 we found some suggestive relationships between the measures we observed. Specifically, the MSLQ anxiety factor predicted higher reported difficulty (r=.42) on the post practice questionnaire (difficulty reports also correlated negatively (-.4) with performance during practice). Also, some individual post practice difficulty items marginally correlated negatively with amount of time spend practicing. Another suggestive result was that MSLQ anxiety was correlated (.31) with rote type strategic behavior questions on the MSLQ while rote strategic behaviors were also correlated with post practice difficulty (.29). Further analysis will focus on how specific student behaviors (e.g. the amount of practice trials that end with timeout as compared to a guess) is associated with MSLQ responses, post survey responses, or learning and performance measures.

Dificulty and Strategy Study 1

Experiment 2 will attempt to react to these sorts of correlations by investigating how difficulty may account for lack of persistence and reduced performance. Specifically, my working model of motivation (based on Zimmerman’s self-regulation theory (2008) and Bandura’s Triadic theory (Zimmerman, 1989)) highlights the importance of perceived task difficulty in determining efficacy and value, which in turn determine planning, intention and action (which produces a cycle as strategy is applied to the task and difficulty is again evaluated with each action cycle). Given this model, the difficulty factor is notable since it should feed back to influence value (in the case difficulty is too easy) or efficacy (in the case of difficulty being too hard) and this has the potential to collapse the whole process (resulting in a student who quits or persists without engagement) if difficulty is too high or too low. Using all 7-8 sections of Chinese I and II (~80-100 students) over 2 lessons, this study (beginning Fall semester) will use 3 levels of initial difficulty set by fixing the initial learning value in the adaptive student model (setting initial learning high creates a high difficulty condition, etc.). Students will be given a range of persistence as acceptable for completion of the in class assignment (from 20 to 40 minutes will be counted for course credit for each lesson).

Hypothesis

A first main effect we expect to see is that higher reported difficulty will foreshadow reduced efficacy and, in turn, result in reduced persistence and a reduction in strategy use that will be accompanied by decreases in the fluency of performance in the tutor. A second main effect we expect to see is that reports of low difficulty will foreshadow reduced value ratings and also result in reduced persistence and performance. We predict an inverted U shaped plot of the data showing how those students that are nearest their reported optimal difficulty showed the most persistence and maintain high levels of performance across time. We will attempt to directly relate the current adaptive model that predicts the optimal difficulty to maximize learning with the optimal perceived difficulty according to student reports.

Independent Variables

In addition to pre and post motivational surveys, we will also include self-evaluative assessments that periodically (every 1 minute) ask the student’s opinion about their experience. In this recording the goal is to show that self reports of difficulty, efficacy, value, and strategy use will predict persistence and performance during the practice.

Dependent Variables

Depending on the analysis, some of the measures are either dependent or independent. Persistence and reaction to the tutoring are the main dependent variables.

Results

Forthcoming

Explanation

Forthcoming

Dificulty and Strategy Study 2

Because difficulty and strategy use reports may interact (Pavlik dissertation, 2005 and Experiment 1 above)(Pavlik Jr., 2005), this experiment will manipulate whether or not students receive strategy advice and also allow some students to have the option of a difficulty control slider that allows students to regulate their own level of challenge during practice (this indicates a 2x2 between subjects design). Self regulation and strategy use will be manipulated by the instructions given to students. One group of students will receive instructions to just complete the task and do extra practice if they want. The experimental group will in contrast receive strategic advice about the process of forming mnemonic links during drill practice. This manipulation has been shown to improve performance significantly in the lab (Pavlik Jr., 2007a, 2007b) and is based on well established principles of mnemonic technique (Atkinson, 1975).

Hypothesis

This experiment will help determine if there are further gains to persistence or performance measures from teaching strategies and providing the difficulty slider. Most importantly, we hope to show in all conditions that strategy use is higher for those subjects who report optimal perceived difficulty. Providing the difficulty slider should allow students to better calibrate their motivationally optimal difficulty, and should therefore enhance the use of strategies in both conditions.

Independent Variables

In addition to pre and post motivational surveys, we will also include self-evaluative assessments that periodically (every 1 minute) ask the student’s opinion about their experience. In this recording the goal is to show that self reports of difficulty, efficacy, value, and strategy use will predict persistence and performance during the practice.

Dependent Variables

Depending on the analysis, some of the measures are either dependent or independent. Persistence and reaction to the tutoring are the main dependent variables.

Results

Forthcoming

Explanation

Forthcoming

Contributions

Forthcoming, main planned contribution is a better understanding of how experienced difficulty influences and is influenced by motivational variables. Additionally we will be trying to understand how these relations are mediated by strategy use.

References

• Atkinson, R. C. (1975). Mnemotechnics in second-language learning. American Psychologist, 30, 821-828.
• Pavlik Jr., P. I. (2005, dissertation). The microeconomics of learning: Optimizing paired-associate memory. Dissertation Abstracts International: Section B: The Sciences and Engineering, 66, 5704.
• Pavlik Jr., P. I. (2007a). Timing is an order: Modeling order effects in the learning of information. In F. E., Ritter, J. Nerb, E. Lehtinen & T. O'Shea (Eds.), In order to learn: How order effects in machine learning illuminate human learning (pp. 137-150). New York: Oxford University Press.
• Pavlik Jr., P. I. (2007b). Understanding and applying the dynamics of test practice and study practice. Instructional Science, 35, 407-441.
• Pintrich, P. R., & de Groot, E. V. (1990). Motivational and self-regulated learning components of classroom academic performance. Journal of Educational Psychology, 82, 33-40.

Future Plans

Incorporate results in current systems to improve student motivation.