DiBiano Personally Relevant Algebra Problems
Contents
Robust Learning in Culturally and Personally Relevant Algebra Problem Scenarios
Candace DiBiano, Anthony Petrosino, Jim Greeno, and Milan Sherman
Summary Tables
| PIs | Candace DiBiano & Anthony Petrosino |
| Other Contributers |
|
Pre Study
| Study Start Date | 09/01/08 |
| Study End Date | 12/15/08 |
| Study Site | Austin ISD, Texas |
| Number of Students | N = 200 |
| Average # of hours per participant | 3 hrs. |
Full Study
| Study Start Date | 2/1/09 or 9/1/09 |
| Study End Date | 6/1/09 or 12/15/09 |
| LearnLab Site | TBD |
| LearnLab Course | Algebra |
| Number of Students | N = 60-90 |
| Average # of hours per participant | 1 hr |
| Data in DataShop | n/a |
Abstract
In the original development of the PUMP Algebra Tutor (PAT), teachers had designed the algebra problem scenarios to be "culturally and personally relevant to students" (Koedinger, 2001). However, observations and discussions with teachers in Austin ISD suggest that the problem scenarios are disconnected from the lives of typical urban students. This study will examine whether and the mechanisms by which cultural and personal familiarity with problem scenario context affect comprehension and robust learning. We will use the medium of Cognitive Tutor Algebra for the in-vivo portion of this study, but our aim is not to improve the quality of the software’s problem scenarios. It is instead to study how student diversity affects cognition, motivation, and learning, by using the power of a computer system that has the ability to do what classroom teachers cannot – personalize each problem to the background and interests of each individual student.
The research will begin in Fall of 2008 with a study of the cultural and personal interests of urban students in Austin ISD. Freshman algebra students will be surveyed and interviewed over their interests, such as sports, music, movies, etc., and the results of this study will be used to rewrite the algebra problem scenarios in one section of the Cognitive Tutor software. Also during the Fall of 2008, a smaller group of students at a Pittsburgh Learnlab site will be surveyed on their interests to ensure they are comparable to Austin students. In the Spring of 2009 at the Pittsburgh Learnlab site (*this may be delayed until Fall 2009), the Cognitive Tutor software will be programmed to give students an interests survey, and then select problem scenarios that match user interests. The resulting robust learning, measured by curriculum progress and mastery of knowledge components, will be analyzed with a 3-group design to measure the effects of the personalization.
This research will be integrated with a study incorporating think-aloud protocols of students solving algebra problems in familiar and unfamiliar contexts to examine how personal relevance and cultural familiarity interacts with conceptual difficulty in interpreting the problem.
Background and Significance
This research direction was initiated by the observation of classrooms in Austin, Texas using the Cognitive Tutor Algebra I software, as well as discussions with teachers that had implemented this software at some point in their teaching career. Teacher complaints were consistently centered not around the interface, the feedback, or the cognitive model of the software, but on the problem scenarios. Teachers explained that their urban students found problems about harvesting wheat “silly,” “dry,” and irrelevant. Teachers also complained that some of the vocabulary words in the Cognitive Tutor problem scenarios (one example was the word "greenhouse") confused their students because urban freshman do not typically discuss these topics in their everyday speech. It’s important to note that as part of the development of the PUMP Algebra Tutor (PAT), teachers had designed problems to be "culturally and personally relevant to students" (Koedinger, 2001). This research is designed to empirically test the claim that the cultural and personal relevance of problem scenarios affects robust learning.
Research Questions
- How will robust learning be affected when personalization through culturally relevant problem scenarios is implemented instead of the current problem scenarios in the Cognitive Tutor Algebra I software?
Independent variables
This experiment will manipulate level of personalization through two treatment groups:
- Students recieve current Cognitive Tutor Algebra problems
- Students receive matched culturally relevant Cognitive Tutor Algebra problems personalized according to student interest survey
| Treatment | Example Problem | Received By |
| Normal Cognitive Tutor Algebra problem scenarios | A skier noticed that she can complete a run in about 30 minutes. A run consists of riding the ski lift up the hill, and skiing back down. If she skiis for 3 hours, how many runs will she have completed? | 25-30 randomly-assigned Algebra I students at Learnlab site |
| Culturally relevant personalized problem scenarios | (student selects personal interest in T.V. shows, cultural survey/interview shows strong interest among urban youth in reality shows)
You noticed that the reality shows you watch on T.V. are all 30 minutes long. If you’ve been watching reality shows for 3 hours, how many have you watched? |
25-30 randomly-assigned Algebra I students at Learnlab site |
Hypothesis
Students in the treatment with culturally and personally relevant problem scenarios will show improved performance in terms of some measures of robust learning as a result of two factors:
- Increased intrinsic motivation (such as with the REAP Tutor study)
- Formation of a more detailed and meaningful situation model (Nathan, Kintsh, & Young, 1992).
Dependent variables
Robust learning will principally be measured through:
- Curriculum progress through the Cognitive Tutor software:
- The students’ progress through the curriculum will measure long-term retention of the knowledge components mastered during the portions of the software where they are given personally relevant problem scenarios.
- The students’ progress through the curriculum will measure ability to transfer learning from personally relevant problem scenarios to normal problem scenarios and abstract (symbolic) problem scenarios.
- The students’ progress through the curriculum will measure accelerated future learning by reflecting the latency in mastering knowledge components that build on the knowledge components affected by the culturally relevant problem scenarios (such as quadratic equations building on linear equations).
- Mastery of knowledge components in the Cognitive Tutor software:
- Mastery of individual knowledge components will be a related, but finer-grained measure of how relevant vs. non-relevant problem scenarios affect robust learning.
- Classroom-based assessments may also be used to evaluate robust learning.
Method
This experiment will begin in the Fall of 2008 with a study of student cultural interests. An interests survey will be administered to high school classes in Austin ISD that contain a high proportion of diverse students. The same interests survey will also be administered to the students at the selected Leanlab site, to ensure that their interests are comparable to those of the students in Austin. Structured in-depth interviews relating to student interests will be conducted with around fifteen of the Austin ISD students. Based on the results of the survey and interviews, culturally relevant problem scenarios that correspond to current problem scenarios in Cognitive Tutor Algebra I will be formulated for Section 9, Linear Models and Two Quadrant Graphs (for a Spring 2009 implementation) or Section 3, Linear Models and First Quadrant Graphs (for a Fall 2009 implementation). Approximately 30 problem scenarios from the selected section will be replaced, with 4-5 variations on each problem scenario that correspond to different student interests, in order to obtain personalization. I will write these problem scenarios while consulting with Jim Greeno and Milan Sherman; they will have the same underlying mathematics as the original Cognitive Tutor problems, with changes to the objects or nouns (what the problem is about) and the pronouns (who the problem is about). See the table above for an example of how these two changes might occur.
The culturally relevant problem scenarios will be reviewed by Algebra I teachers, and then by students. In a pilot study in Austin ISD, approximately 40 Algebra I students will rate their understanding and impression of the newly created questions. Problem scenarios that students have difficulties or issues with will be reworked. Also during this pilot study, the researcher will conduct audio-taped think-aloud protocols with each student as they solve 2-3 personally relevant algebra problem scenarios and 2-3 algebra problem scenarios with unfamiliar contexts. This will also aid the researcher in performing a difficulty factors assessment. I have positioned myself as an Algebra I teaching assistant in a diverse Austin ISD school for Fall or 2009 in order to complete this phase of the study.
The new problem scenarios will then be integrated into the Cognitive Tutor Algebra software in Spring 2009 with the cooperation of Carnegie Learning. Once the new problem scenarios have been placed into the software, they will be used in an in vivo experiment at a Learnlab school site in Pittsburgh by approximately 25-30 randomly-assigned students in the Spring 2009 or Fall of 2009 semester. An additional 25-30 randomly-assigned students will receive the regular problem scenarios. See table above for a description of the two treatment groups in this study.
In addition, informal interviews will be conducted with students at the University of Pittsburgh, including thinking-aloud protocols obtained as they solve word problems with texts that differ in the degree of their cultural relevance to the students. These protocols will be analyzed to identify components of students’ understanding (i.e., their situation models), and to relate these to cultural relevance and familiarity.
To summarize, the experiment will have the following progression: (1) Survey of student interests administered in Austin ISD and Learnlab site (2) Based on survey data, structured interviews with students are conducted in Austin ISD (3) Culturally relevant problem scenarios are written by me and reviewed by teachers (4) Culturally relevant problem scenarios are tested for understanding and as part of a think-aloud protocols during a student pilot study in Austin ISD (5) One Cognitive Tutor Agelbra unit replaced at a Learnlab site with 2-treatment setup & think-aloud protocols conducted at University of Pittsburgh
Explanation
In development
References
Clark, R. C. & Mayer, R. E. (2003). E-Learning and the Science of Instruction. Jossey-Bass/Pfeiffer.
Cordova, D. I. & Lepper, M. R. (1996). Intrinsic Motivation and the Process of Learning: Beneficial Effects of Contextualization, Personalization, and Choice. Journal of Educational Psychology, 88(4), 715-730.
Koedinger, K. R. (2001). Cognitive tutors as modeling tool and instructional model. In Forbus, K. D. & Feltovich, P. J. (Eds.) Smart Machines in Education: The Coming Revolution in Educational Technology. Menlo Park, CA: AAAI/MIT Press.
Nathan, M., Kintsch, W., & Young, E. (1992). A theory of algebra-word-problem comprehension and its implications for the design of learning environments. Cognition and Instruction, 9(4), 329-389.
