DiBiano Personally Relevant Algebra Problems - Revision history

Mbett: Reverted edits by Kamearobinson (Talk); changed back to last version by Dibiano

2011-08-31T15:37:47Z

Reverted edits by Kamearobinson (Talk); changed back to last version by Dibiano

← Older revision		Revision as of 15:37, 31 August 2011
Line 168:		Line 168:

	[[Category:Study]]		[[Category:Study]]


	~~[http://editingwritingservices.org/hesitating.php creative writing services]~~

Kamearobinson at 09:56, 31 August 2011

2011-08-31T09:56:20Z

← Older revision		Revision as of 09:56, 31 August 2011
Line 168:		Line 168:

	[[Category:Study]]		[[Category:Study]]


			[http://editingwritingservices.org/hesitating.php creative writing services]

Dibiano: /* Theoretical Framework */

2010-11-16T21:51:13Z

Theoretical Framework

← Older revision		Revision as of 21:51, 16 November 2010
Line 58:		Line 58:
	This study is situated in the new “Motivation and Metacogntion” thrust. The foundation of this study is that relevance of problem scenarios affects robust learning through increased intrinsic motivation (Cordova & Lepper, 1996). If learners that have the cognitive capacity to solve algebra story problems, enhancing motivation may increase their likelihood to exert effort to make sense of the scenarios by forming a more elaborated and better connected situation and problem models (Nathan, Kintsch, & Young, 1992), thus encouraging generative processing (Mayer, 2011). Mayer (2011) states the personalization principle as “People learn better when the instructor uses conversational style rather than formal style” (p. 70). Here, we are use the PSLC’s modified version of this principle, which states “Matching up the features of an instructional component with students' personal interests, experiences, or typical patterns of language use, will lead to more robust learning through increased motivation, compared to when instruction is not personalized.” This is related to what Mayer (2011) refers to as the “Anchoring” principle.		This study is situated in the new “Motivation and Metacogntion” thrust. The foundation of this study is that relevance of problem scenarios affects robust learning through increased intrinsic motivation (Cordova & Lepper, 1996). If learners that have the cognitive capacity to solve algebra story problems, enhancing motivation may increase their likelihood to exert effort to make sense of the scenarios by forming a more elaborated and better connected situation and problem models (Nathan, Kintsch, & Young, 1992), thus encouraging generative processing (Mayer, 2011). Mayer (2011) states the personalization principle as “People learn better when the instructor uses conversational style rather than formal style” (p. 70). Here, we are use the PSLC’s modified version of this principle, which states “Matching up the features of an instructional component with students' personal interests, experiences, or typical patterns of language use, will lead to more robust learning through increased motivation, compared to when instruction is not personalized.” This is related to what Mayer (2011) refers to as the “Anchoring” principle.

	The construct through which personalization enhances intrinsic motivation is through ~~increase~~ personal interest (also called individual interest). Personal interest is considered to be stable, enduring preferences that individual learners bring with them to different situations (Anderman & Anderman, 2010). Interest promotes more effective processing of information and greater cognitive engagement. Students who have high interest may be more likely to relate new knowledge to prior knowledge and form more connections between ideas. They also may be more likely to generate inferences, examples and applications relating to the subject area they are trying to learn (Ormrod, 2008).		The construct through which personalization enhances intrinsic motivation is through increased personal interest (also called individual interest). Personal interest is considered to be stable, enduring preferences that individual learners bring with them to different situations (Anderman & Anderman, 2010). Interest promotes more effective processing of information and greater cognitive engagement. Students who have high interest may be more likely to relate new knowledge to prior knowledge and form more connections between ideas. They also may be more likely to generate inferences, examples and applications relating to the subject area they are trying to learn (Ormrod, 2008).

	=== Pilot Study===		=== Pilot Study===

Dibiano at 21:50, 16 November 2010

2010-11-16T21:50:14Z

Dibiano at 21:47, 16 November 2010

2010-11-16T21:47:08Z

Show changes

Dibiano at 19:42, 26 July 2010

2010-07-26T19:42:13Z

Dibiano at 19:38, 26 July 2010

2010-07-26T19:38:11Z

← Older revision		Revision as of 19:38, 26 July 2010
Line 11:		Line 11:
	\|}		\|}
	<br>		<br>
	'' Pre Study ''		'' Pre Study (Think-Alouds) ''
	{\| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"		{\| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
	\| '''Study Start Date''' \|\| ~~09/01/08~~		\| '''Study Start Date''' \|\| September 2008
	\|-		\|-
	\| '''Study End Date''' \|\| ~~4/15/10~~		\| '''Study End Date''' \|\| May 2009
	\|-		\|-
	\| '''Study Site''' \|\| Austin, ~~Texas & Pittsburgh, PA~~		\| '''Study Site''' \|\| Austin, TX
	\|-		\|-
	\| '''Number of Students''' \|\| ''N'' = ~~125~~		\| '''Number of Students''' \|\| ''N'' = 29
	\|-		\|-
	\| '''Average # of hours per participant''' \|\| 3 hrs.		\| '''Average # of hours per participant''' \|\| 3 hrs.
Line 27:		Line 27:
	'' Full Study ''		'' Full Study ''
	{\| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"		{\| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
	\| '''Study Start Date''' \|\| ~~9/1/08~~		\| '''Study Start Date''' \|\| October 2009
	\|-		\|-
	\| '''Study End Date''' \|\| ~~4/15/10~~		\| '''Study End Date''' \|\| April 2010
	\|-		\|-
	\| '''LearnLab Site''' \|\| Hopewell High		\| '''LearnLab Site''' \|\| Hopewell High
Line 35:		Line 35:
	\| '''LearnLab Course''' \|\| Algebra		\| '''LearnLab Course''' \|\| Algebra
	\|-		\|-
	\| '''Number of Students''' \|\| ''N'' = ~~125~~		\| '''Number of Students''' \|\| ''N'' = 111
	\|-		\|-
	\| '''Average # of hours per participant''' \|\| 3 hr		\| '''Average # of hours per participant''' \|\| 3 hr

Dibiano at 19:36, 26 July 2010

2010-07-26T19:36:14Z

Show changes

Dibiano at 21:01, 26 August 2008

2008-08-26T21:01:44Z

← Older revision		Revision as of 21:01, 26 August 2008
Line 95:		Line 95:
	*Hint-seeking and reading behavior in Cognitive Tutor software		*Hint-seeking and reading behavior in Cognitive Tutor software
	*Time on task in Cognitive Tutor software		*Time on task in Cognitive Tutor software
	*Questionairre asking how interesting ~~and fun~~ students found problems in the affected unit		*Questionairre asking how interesting students found problems in the affected unit

	=== Method ===		=== Method ===
Line 112:		Line 112:
	(3) Culturally relevant problem scenarios are written by me and reviewed by teachers		(3) Culturally relevant problem scenarios are written by me and reviewed by teachers
	(4) Culturally relevant problem scenarios are tested for understanding and as part of a [[think-aloud data\|think-aloud protocols]] during a student pilot study		(4) Culturally relevant problem scenarios are tested for understanding and as part of a [[think-aloud data\|think-aloud protocols]] during a student pilot study
	(5) One [[cognitive tutor\|Cognitive Tutor]] ~~Agelbra~~ unit replaced at a Learnlab site with 3-treatment setup & [[think-aloud data\|think-aloud protocols]] conducted at University of Pittsburgh		(5) One [[cognitive tutor\|Cognitive Tutor]] Algebra unit replaced at a Learnlab site with 3-treatment setup & [[think-aloud data\|think-aloud protocols]] conducted at University of Pittsburgh

	=== Explanation ===		=== Explanation ===

	This ~~research~~ is situated ~~within~~ the new "~~Motivation and Metacognition~~" ~~thrust~~.		This study is situated in the new “Motivation and Metacogntion’ thrust. The foundation of this study is that relevance of problem scenarios affects robust learning during the formation situation models, defined as mental representation of relationships, actions, and events in a problem (Nathan, Kintsch, & Young, 1992), as well through intrinsic motivation (Cordova & Lepper, 1996). Our hypothesis is that personalized problems would cause students to create more detailed and meaningful situation models through enhanced problem comprehension ad implicit problem knowledge. This would in turn affect the topology of the learning event space and/or path choices, causing students to use different strategies or paths (“blue-line” vs. “red-line”) as relevant problems are more likely to help students to encode deep, relevant features and/or avoid encoding shallow, irrelevant features. Another facet of this hypothesis is that personalized problems would enhance intrinsic motivation, which would increase focus of attention on the problem, contributing both to the formation of detailed situation models as well as more general enhancement of engagement and time on task (relating to "path effects").

	=== References ===		=== References ===

Dibiano at 03:41, 26 August 2008

2008-08-26T03:41:05Z

← Older revision		Revision as of 03:41, 26 August 2008
Line 87:		Line 87:
	=== Dependent variables ===		=== Dependent variables ===
	[[Robust learning]] will be measured through:		[[Robust learning]] will be measured through:
	* '''''[[Normal_post-test\|Normal Post-test]]''''' measuring ~~near-transfer and~~ [[transfer]] of learning.		* '''''[[Normal_post-test\|Normal Post-test]]''''' measuring [[transfer]] of learning to different problem contexts (including abstract problems).
	*'''''Delayed Post-test''''' ~~measiring~~ [[long-term retention]]		*'''''Delayed Post-test''''' measuring [[long-term retention]]
	* '''''Curriculum progress''''' and '''''Mastery of knowledge components''''' in the [[cognitive tutor\|Cognitive Tutor]] software, including in subsequent units:		* '''''Curriculum progress''''' and '''''Mastery of knowledge components''''' in the [[cognitive tutor\|Cognitive Tutor]] software, including in subsequent units:
	**The students’ progress through the knowledge components in the curriculum will measure [[accelerated future learning]] by reflecting the latency in mastering knowledge components that build on the knowledge components affected by the culturally relevant problem scenarios ~~(such as quadratic equations building on linear equations)~~.		**The students’ progress through the knowledge components in the curriculum will measure [[accelerated future learning]] by reflecting the latency in mastering knowledge components and curriculum sections that build on the knowledge components and curriculum sections affected by the culturally relevant problem scenarios.

	'''Intrinsic Motivation''' will be measured through:		'''Intrinsic Motivation''' will be measured through:

← Older revision		Revision as of 21:50, 16 November 2010
Line 64:		Line 64:
	The first stage of this research began in Fall of 2008 with a pilot study of personalization at an "Academically Unacceptable" school in Texas (75% free/reduced lunch). Twenty-four freshman algebra students were interviewed about their out-of-school interests, such as sports, music, movies, etc., and were also asked to describe how they use mathematics in their everyday lives. These interviews were audio recorded, and were used to write each student “personalized” algebra story problems. The research questions being investigated were:		The first stage of this research began in Fall of 2008 with a pilot study of personalization at an "Academically Unacceptable" school in Texas (75% free/reduced lunch). Twenty-four freshman algebra students were interviewed about their out-of-school interests, such as sports, music, movies, etc., and were also asked to describe how they use mathematics in their everyday lives. These interviews were audio recorded, and were used to write each student “personalized” algebra story problems. The research questions being investigated were:

	1) What is the impact of personalizing algebra story problems to individual student experiences, in terms of strategy use, language comprehension, and students’ epistemological frames about mathematical activity? (qualitative)		* What is the impact of personalizing algebra story problems to individual student experiences, in terms of strategy use, language comprehension, and students’ epistemological frames about mathematical activity? (qualitative)
	2) How does personalizing algebra story problems to individual experiences impact student performance, when compared to their performance on normal story problems from the Cognitive Tutor curriculum with the same underlying structure? (quantitative)
			* How does personalizing algebra story problems to individual experiences impact student performance, when compared to their performance on normal story problems from the Cognitive Tutor curriculum with the same underlying structure? (quantitative)

	A problem set containing five algebra problems on linear functions was written for each student; two of these were story problems that were personalized to the ways in which the individual student described using mathematics in their everyday life during their initial interview. The problem set also contained normal story problems from the Cognitive Tutor curriculum, completely abstract symbolic equations, story problems that contained symbolic equations, and story problems with simplified language and general referents (“generic” story problems). Each problem had four parts – the first two parts were “Result Unknowns” or “concrete cases” (i.e. solve for y given this x), and the fourth and final part was a “Start Unknown” (i.e. solve for x given this y). For normal, personalized, and generic problems, the third part of each problem asked students to write a general symbolic equation or “algebra rule” representing the story. For normal story problems that already contained equations, students were asked to interpret the parameters in terms of the story. For completely abstract symbolic problems, students were asked to write a story that could go with the equation.		A problem set containing five algebra problems on linear functions was written for each student; two of these were story problems that were personalized to the ways in which the individual student described using mathematics in their everyday life during their initial interview. The problem set also contained normal story problems from the Cognitive Tutor curriculum, completely abstract symbolic equations, story problems that contained symbolic equations, and story problems with simplified language and general referents (“generic” story problems). Each problem had four parts – the first two parts were “Result Unknowns” or “concrete cases” (i.e. solve for y given this x), and the fourth and final part was a “Start Unknown” (i.e. solve for x given this y). For normal, personalized, and generic problems, the third part of each problem asked students to write a general symbolic equation or “algebra rule” representing the story. For normal story problems that already contained equations, students were asked to interpret the parameters in terms of the story. For completely abstract symbolic problems, students were asked to write a story that could go with the equation.
Line 116:		Line 117:
	Students in the treatment with [[personalization\|personalized]] problem scenarios will:		Students in the treatment with [[personalization\|personalized]] problem scenarios will:

	H1) Demonstrate higher levels of correct performance in Section 5		H1) Demonstrate higher levels of correct performance in Section 5

	H2) Show improved “time on task” and fewer instances of “gaming the system” in Section 5		H2) Show improved “time on task” and fewer instances of “gaming the system” in Section 5

	H3) Show improvement on some measures of [[robust learning]], as measured by pre/delayed post differences and by performance in subsequent sections.		H3) Show improvement on some measures of [[robust learning]], as measured by pre/delayed post differences and by performance in subsequent sections.

Line 132:		Line 135:
	In order to test this hypothesis, a logistic regression model was formulated with the following properties. The unit of analysis was one student solving one part of one problem.		In order to test this hypothesis, a logistic regression model was formulated with the following properties. The unit of analysis was one student solving one part of one problem.

	1) Dependent Variable – whether the student got the problem part correct on their first attempt, without asking for a hint.		* Dependent Variable – whether the student got the problem part correct on their first attempt, without asking for a hint.
	2) Random Effects – the student ID , the item (linear function underlying the problem), and the problem name (which personalized version student was given, or which set of numbers student was given for result and start unknowns)		* Random Effects – the student ID , the item (linear function underlying the problem), and the problem name (which personalized version student was given, or which set of numbers student was given for result and start unknowns)
	3) Fixed Effects – Condition (whether the student was in the experimental or control group) and what knowledge component was covered by the problem part		* Fixed Effects – Condition (whether the student was in the experimental or control group) and what knowledge component was covered by the problem part

	Each of these effects significantly improved the model. Interactions did not significantly improve the model. The main effect for the treatment (personalization) was statistically significant at the 5% level. Personalization had a positive overall effect on student performance. The size of the overall impact of personalization on performance was around 5.3%. If a student had a 50% base chance of getting a problem correct on the first attempt, personalization would increase that chance to 55.3%.		Each of these effects significantly improved the model. Interactions did not significantly improve the model. The main effect for the treatment (personalization) was statistically significant at the 5% level. Personalization had a positive overall effect on student performance. The size of the overall impact of personalization on performance was around 5.3%. If a student had a 50% base chance of getting a problem correct on the first attempt, personalization would increase that chance to 55.3%.

← Older revision		Revision as of 19:42, 26 July 2010
Line 66:		Line 66:
	\| '''Treatment'''\|\| '''Example Problem''' \|\| '''Received By'''		\| '''Treatment'''\|\| '''Example Problem''' \|\| '''Received By'''
	\|-		\|-
	\| Normal Cognitive Tutor Algebra problem scenarios \|\| A skier noticed that she can complete a run in about 30 minutes. A run consists of riding the ski lift up the hill, and skiing back down. If she skiis for 3 hours, how many runs will she have completed? \|\| ~~25-30~~ randomly-assigned Algebra I students at Learnlab site		\| Normal Cognitive Tutor Algebra problem scenarios \|\| A skier noticed that she can complete a run in about 30 minutes. A run consists of riding the ski lift up the hill, and skiing back down. If she skiis for 3 hours, how many runs will she have completed? \|\| 54 randomly-assigned Algebra I students at Learnlab site
	\|-		\|-
	\| Relevant [[personalization\|personalized]] problem scenarios \|\| (student selects personal interest in T.V. shows, cultural survey/interview shows strong interest among urban youth in reality shows)		\| Relevant [[personalization\|personalized]] problem scenarios \|\| (student selects personal interest in T.V. shows, cultural survey/interview shows strong interest among urban youth in reality shows)
	You noticed that the reality shows you watch on T.V. are all 30 minutes long. If you’ve been watching reality shows for 3 hours, how many have you watched?		You noticed that the reality shows you watch on T.V. are all 30 minutes long. If you’ve been watching reality shows for 3 hours, how many have you watched?
	\|\| ~~110~~ randomly-assigned Algebra I students at Learnlab site		\|\| 57 randomly-assigned Algebra I students at Learnlab site
	\|}		\|}
	<BR>		<BR>
Line 93:		Line 93:
	=== Method ===		=== Method ===

	This experiment began in the Fall of 2008 with a study of student interests. An interests survey was administered to high school classes in Austin ISD that contain a high proportion of diverse students, as well as at a Pittsburgh Learnlab. Structured in-depth interviews relating to student interests were conducted with around 29 of the surveyed students. Based on the results of the survey and interviews, culturally relevant problem scenarios that correspond to current problem scenarios in [[cognitive tutor\|Cognitive Tutor]] Algebra I were formulated for Section 5, Linear Models and Independent Variables. Approximately 27 problem scenarios from the selected section ~~will be~~ replaced, with 4 variations on each problem scenario that correspond to different student interests, in order to obtain [[personalization]]. I wrote these problem scenarios while consulting with Jim Greeno and Milan Sherman; they ~~will have~~ the same underlying mathematics as the original [[cognitive tutor\|Cognitive Tutor]] problems, with changes to the objects or nouns (what the problem is about) and the pronouns (who the problem is about). See the table above for an example of how these two changes ~~might occur~~.		This experiment began in the Fall of 2008 with a study of student interests. An interests survey was administered to high school classes in Austin ISD that contain a high proportion of diverse students, as well as at a Pittsburgh Learnlab. Structured in-depth interviews relating to student interests were conducted with around 29 of the surveyed students. Based on the results of the survey and interviews, culturally relevant problem scenarios that correspond to current problem scenarios in [[cognitive tutor\|Cognitive Tutor]] Algebra I were formulated for Section 5, Linear Models and Independent Variables. Approximately 27 problem scenarios from the selected section were replaced, with 4 variations on each problem scenario that correspond to different student interests, in order to obtain [[personalization]]. I wrote these problem scenarios while consulting with Jim Greeno and Milan Sherman; they had the same underlying mathematics as the original [[cognitive tutor\|Cognitive Tutor]] problems, with changes to the objects or nouns (what the problem is about) and the pronouns (who the problem is about). See the table above for an example of how these two changes occured.

	The culturally relevant problem scenarios were reviewed by two master Algebra I teachers. In a pilot study, 24 Algebra I students participated in [[think-aloud data\|think-aloud protocols]] where they solved five story problems with varying degrees of relevancy, that were based on Cognitive Tutor problems. ~~Problem scenarios that students have difficulties or issues with will be reworked.~~		The culturally relevant problem scenarios were reviewed by two master Algebra I teachers. In a pilot study, 24 Algebra I students participated in [[think-aloud data\|think-aloud protocols]] where they solved five story problems with varying degrees of relevancy, that were based on Cognitive Tutor problems.

	The new problem scenarios were integrated into the [[cognitive tutor\|Cognitive Tutor]] Algebra software in Summer 2009 with the cooperation of Carnegie Learning. Once the new problem scenarios were placed into the software, they were used in an [[in vivo experiment]] at a Learnlab school site in Pittsburgh by ~~approximately 50-55~~ randomly-assigned students during the 09-10 school year. An additional ~~50-55~~ randomly-assigned students received the regular problem scenarios. See table above for a description of the two treatment groups in this study.		The new problem scenarios were integrated into the [[cognitive tutor\|Cognitive Tutor]] Algebra software in Summer 2009 with the cooperation of Carnegie Learning. Once the new problem scenarios were placed into the software, they were used in an [[in vivo experiment]] at a Learnlab school site in Pittsburgh by 57 randomly-assigned students during the 09-10 school year. An additional 54 randomly-assigned students received the regular problem scenarios. See table above for a description of the two treatment groups in this study.

	To summarize, the experiment had the following progression:		To summarize, the experiment had the following progression:
	(1) Survey (paper & online) of student interests administered in Austin ISD and Learnlab site		(1) Survey (paper & online) of student interests administered in Austin ISD and Learnlab site
	(2) Based on survey data, structured interviews on students' out-~~o9f~~-school interests were conducted		(2) Based on survey data, structured interviews on students' out-of-school interests were conducted
	(3) Based on interest interview, 24 students participated in think-alouds where they each solved 5 problems with different degrees of relevancy.		(3) Based on interest interview, 24 students participated in think-alouds where they each solved 5 problems with different degrees of relevancy
	(4) Relevant problem scenarios for Section 5 were written by Candace Walkington & Milan Sherman and reviewed by 2 master algebra teachers		(4) Relevant problem scenarios for Section 5 were written by Candace Walkington & Milan Sherman and reviewed by 2 master algebra teachers
	(5) One [[cognitive tutor\|Cognitive Tutor]] Algebra unit replaced at a Learnlab site with randomized control (in-sequence) setup		(5) One [[cognitive tutor\|Cognitive Tutor]] Algebra unit was replaced at a Learnlab site with randomized control (in-sequence) setup, N=111.

	=== Explanation ===		=== Explanation ===