Harnessing what you know - Revision history

Koedinger: /* The instructional unit as the knowledge component */

2009-05-29T14:46:41Z

The instructional unit as the knowledge component

← Older revision		Revision as of 14:46, 29 May 2009
Line 43:		Line 43:

	===The instructional unit as the knowledge component===		===The instructional unit as the knowledge component===
	The first knowledge component treated each unit as a separate knowledge component. Because we were initially interested in far transfer, we included two units: translational and rotational kinematics. We also included a third unit, translational dynamics, as a control case. Translational dynamics occurred after translational kinematics, but before rotational kinematics. Therefore, we would expect the learning curves for translational dynamics to fall somewhere between translational and rotational kinematics. The learning curves, over three opportunities, can be found in Figure 3.<br><br>		The first knowledge component analysis treated each unit as a separate knowledge component. Because we were initially interested in far transfer, we included two units: translational and rotational kinematics. We also included a third unit, translational dynamics, as a control case. Translational dynamics occurred after translational kinematics, but before rotational kinematics. Therefore, we would expect the learning curves for translational dynamics to fall somewhere between translational and rotational kinematics. The learning curves, over three opportunities, can be found in Figure 3.<br><br>

	[[Image:KCUNIT.jpg\|Figure 3. A using the entire unit as a single knowledge component.]]<br><br>		[[Image:KCUNIT.jpg\|Figure 3. A using the entire unit as a single knowledge component.]]<br><br>

Bobhaus at 17:32, 14 May 2009

2009-05-14T17:32:58Z

Show changes

Bobhaus: /* Harnessing what you know: The role of analogy in robust learning */

2009-05-14T17:25:05Z

Harnessing what you know: The role of analogy in robust learning

← Older revision		Revision as of 17:25, 14 May 2009
Line 3:		Line 3:

	'''Abstract'''. Knowledge transfer is a core assumption built into the pedagogy of most educational programs from K-12 to college. It is assumed that the material learned in the fourth week of the course is retained and transfers to material taught in the eighth week of the course. This is particularly true for highly structured courses such as physics; however, the empirical literature on learning suggests that far transfer is much more difficult than traditional pedagogy assumes (for reviews, see Bransford, Brown, & Cocking, 2000; Bransford & Schwartz, 1999; Gick & Holyoak, 1983). The goal of the present paper is to reconcile these apparently incompatible beliefs. Toward that end, we will use a repository of data, taken from the Physics LearnLab, to argue that the level of granularity of the constituent knowledge components affects the detection of to transfer from one domain to another.<br><br>		'''Abstract'''. Knowledge transfer is a core assumption built into the pedagogy of most educational programs from K-12 to college. It is assumed that the material learned in the fourth week of the course is retained and transfers to material taught in the eighth week of the course. This is particularly true for highly structured courses such as physics; however, the empirical literature on learning suggests that far transfer is much more difficult than traditional pedagogy assumes (for reviews, see Bransford, Brown, & Cocking, 2000; Bransford & Schwartz, 1999; Gick & Holyoak, 1983). The goal of the present paper is to reconcile these apparently incompatible beliefs. Toward that end, we will use a repository of data, taken from the Physics LearnLab, to argue that the level of granularity of the constituent knowledge components affects the detection of to transfer from one domain to another.<br><br>

			==Introduction==
			In well-structured domains, such as math or science, teachers often presume that the contents of one unit will transfer to units taught later in the semester; however, the learning literature is replete with evidence suggesting that transfer, especially ''far transfer'', is difficult to achieve (Detterman, 1993). Do teachers have unrealistic expectations of their students, or are scientists looking in the wrong places to find evidence of far transfer? The primary goal of the present paper is to seek a resolution to this potential contradiction. Toward that end, we will define ''learning'' at multiple levels of granularity and show how different levels of knowledge disaggregation reveal different conclusions about the existence or non-existence of far transfer.<br><br>

			===Knowledge decomposition and learning curves===
			Many domains, such as math, science, and computer programming, assume that knowledge can be decomposed into a partially ordered set of skills or ''knowledge components''. This assumption has been formalized in computational models of human cognition, including ''production rules'' in the ACT-R architecture (Anderson & Lebiere, 1998) and ''chunks'' in the SOAR architecture (Newell, 1990).<br>
			Evidence for the psychological plausibility for knowledge components can be found in the shape of the curve when an individual's performance, which is typically measured as an error rate or the elapsed time, plotted against the opportunities to apply that particular piece of knowledge. These graphs are often referred to as ''learning curves'', and an idealized learning curve monotonically decreases over time. Classic examples of learning curves include memorizing non-sense syllables (Ebbinghaus, 1913), learning how to roll a cigar (Crossman, 1959), and transmitting Morse code (Bryan & Harter, 1897).<br>
			A more contemporary example of a learning curve can be found in the domain of electrodynamics (Hausmann & VanLehn, under review). Students enrolled in a second-semester physic course were asked to solve problems with the Andes Physics Tutor (VanLehn et al., 2005). During an ''in vivo'' experiment (Hausmann & VanLehn, 2007), students were asked to solve four electrodynamics problems, which included calculating the magnitude of an electric force (''F'') that a charged particle (q) experiences when it is located in a region with an electric field (''E''). The relationship between these three quantities is summarized by the following equation: ''F = E

	==Introduction==		==Introduction==

Bobhaus at 17:24, 14 May 2009

2009-05-14T17:24:28Z

Show changes

Bobhaus: /* The instructional unit as the knowledge component */

2009-05-14T17:22:32Z

The instructional unit as the knowledge component

← Older revision		Revision as of 17:22, 14 May 2009
Line 43:		Line 43:

	===The instructional unit as the knowledge component===		===The instructional unit as the knowledge component===
	The first knowledge component treated each unit as a separate knowledge component. Because we were initially interested in far transfer, we included two units: translational and rotational kinematics. We also included a third unit, translational dynamics, as a control case. ~~The reason for including translational dynamics is to control for XXX?~~ Translational dynamics occurred after translational kinematics, but before rotational kinematics. Therefore, we would expect the learning curves for translational dynamics to fall somewhere between translational and rotational kinematics. The learning curves, over three opportunities, can be found in Figure 3.<br><br>		The first knowledge component treated each unit as a separate knowledge component. Because we were initially interested in far transfer, we included two units: translational and rotational kinematics. We also included a third unit, translational dynamics, as a control case. Translational dynamics occurred after translational kinematics, but before rotational kinematics. Therefore, we would expect the learning curves for translational dynamics to fall somewhere between translational and rotational kinematics. The learning curves, over three opportunities, can be found in Figure 3.<br><br>

	[[Image:KCUNIT.jpg\|Figure 3. A using the entire unit as a single knowledge component.]]<br><br>		[[Image:KCUNIT.jpg\|Figure 3. A using the entire unit as a single knowledge component.]]<br><br>

Bobhaus at 17:22, 14 May 2009

2009-05-14T17:22:03Z

Show changes

Bobhaus: /* Discussion */

2009-05-14T17:18:23Z

Discussion

← Older revision		Revision as of 17:18, 14 May 2009
Line 66:		Line 66:
	In the introduction, we pointed out the observation that there is an apparent contradiction between the empirical results investigating far transfer and the assumptions that teachers make within their own classroom. Teachers expect that their students should retain the knowledge components over several weeks, often with many other intervening units of instruction. However, the learning literature on far transfer seems to suggest that it is a rare occasion when knowledge lasts over long retention intervals. <br><br>		In the introduction, we pointed out the observation that there is an apparent contradiction between the empirical results investigating far transfer and the assumptions that teachers make within their own classroom. Teachers expect that their students should retain the knowledge components over several weeks, often with many other intervening units of instruction. However, the learning literature on far transfer seems to suggest that it is a rare occasion when knowledge lasts over long retention intervals. <br><br>
	To resolve the discrepancy between theory and practice, we introduced the hypothesis that the granularity of the assessed knowledge plays a large role in whether transfer is observed or not. For example, when the unit was taken as the knowledge component, then there was absolutely no evidence of transfer. The assistance scores associated with translational kinematics was initially lower (i.e., the first opportunity) than both the translational dynamics and rotational kinematics units. This initial advantage was maintained over fourteen of the sixteen opportunities.<br><br>		To resolve the discrepancy between theory and practice, we introduced the hypothesis that the granularity of the assessed knowledge plays a large role in whether transfer is observed or not. For example, when the unit was taken as the knowledge component, then there was absolutely no evidence of transfer. The assistance scores associated with translational kinematics was initially lower (i.e., the first opportunity) than both the translational dynamics and rotational kinematics units. This initial advantage was maintained over fourteen of the sixteen opportunities.<br><br>
	Because there was no evidence of any sort of transfer, we decomposed the large, unit-size knowledge components into three smaller knowledge components that corresponded to the three broad categories of user-interface elements.~~<br><br>~~		Because there was no evidence of any sort of transfer, we decomposed the large, unit-size knowledge components into three smaller knowledge components that corresponded to the three broad categories of user-interface elements. We repeated this process for the user interface elements that were vectors because the learning curves suggested that there was a drift toward increasing assistance score values. For the most part, the equations and scalar definitions were decreasing as the semester advanced. The vectors were disaggregated into acceleration, velocity, and displacement. These categories were more sensible because they finally corresponded to the concepts that are taught in the physics textbook. <br><br>
	We repeated this process for the user interface elements that were vectors because the learning curves suggested that there was a drift toward increasing assistance score values. For the most part, the equations and scalar definitions were decreasing as the semester advanced. The vectors were disaggregated into acceleration, velocity, and displacement. These categories were more sensible because they finally corresponded to the concepts that are taught in the physics textbook. <br><br>
	Future work will include better understanding why the displacement vector showed such a steep learning curve. At first, students were asking for lots of help and committing many mistakes. However, after making those initial attempts, they seemed to learn how to apply this knowledge component fairly quickly. We also plan to extend our analyses to include the equations that were written. From the student’s perspective, writing equations is the most important part of the course. <br><br>		Future work will include better understanding why the displacement vector showed such a steep learning curve. At first, students were asking for lots of help and committing many mistakes. However, after making those initial attempts, they seemed to learn how to apply this knowledge component fairly quickly. We also plan to extend our analyses to include the equations that were written. From the student’s perspective, writing equations is the most important part of the course. <br><br>

Bobhaus: /* Discussion */

2009-05-14T17:18:05Z

Discussion

← Older revision		Revision as of 17:18, 14 May 2009
Line 64:		Line 64:

	==Discussion==		==Discussion==
	In the introduction, we pointed out the observation that there is an apparent contradiction between the empirical results investigating far transfer and the assumptions that teachers make within their own classroom. Teachers expect that their students should retain the knowledge components over several weeks, often with many other intervening units of instruction. However, the learning literature on far transfer seems to suggest that it is a rare occasion when knowledge lasts over long retention intervals. <br>		In the introduction, we pointed out the observation that there is an apparent contradiction between the empirical results investigating far transfer and the assumptions that teachers make within their own classroom. Teachers expect that their students should retain the knowledge components over several weeks, often with many other intervening units of instruction. However, the learning literature on far transfer seems to suggest that it is a rare occasion when knowledge lasts over long retention intervals. <br><br>
	To resolve the discrepancy between theory and practice, we introduced the hypothesis that the granularity of the assessed knowledge plays a large role in whether transfer is observed or not. For example, when the unit was taken as the knowledge component, then there was absolutely no evidence of transfer. The assistance scores associated with translational kinematics was initially lower (i.e., the first opportunity) than both the translational dynamics and rotational kinematics units. This initial advantage was maintained over fourteen of the sixteen opportunities.<br>		To resolve the discrepancy between theory and practice, we introduced the hypothesis that the granularity of the assessed knowledge plays a large role in whether transfer is observed or not. For example, when the unit was taken as the knowledge component, then there was absolutely no evidence of transfer. The assistance scores associated with translational kinematics was initially lower (i.e., the first opportunity) than both the translational dynamics and rotational kinematics units. This initial advantage was maintained over fourteen of the sixteen opportunities.<br><br>
	Because there was no evidence of any sort of transfer, we decomposed the large, unit-size knowledge components into three smaller knowledge components that corresponded to the three broad categories of user-interface elements.		Because there was no evidence of any sort of transfer, we decomposed the large, unit-size knowledge components into three smaller knowledge components that corresponded to the three broad categories of user-interface elements.<br><br>
	We repeated this process for the user interface elements that were vectors because the learning curves suggested that there was a drift toward increasing assistance score values. For the most part, the equations and scalar definitions were decreasing as the semester advanced. The vectors were disaggregated into acceleration, velocity, and displacement. These categories were more sensible because they finally corresponded to the concepts that are taught in the physics textbook. <br>		We repeated this process for the user interface elements that were vectors because the learning curves suggested that there was a drift toward increasing assistance score values. For the most part, the equations and scalar definitions were decreasing as the semester advanced. The vectors were disaggregated into acceleration, velocity, and displacement. These categories were more sensible because they finally corresponded to the concepts that are taught in the physics textbook. <br><br>
	Future work will include better understanding why the displacement vector showed such a steep learning curve. At first, students were asking for lots of help and committing many mistakes. However, after making those initial attempts, they seemed to learn how to apply this knowledge component fairly quickly. We also plan to extend our analyses to include the equations that were written. From the student’s perspective, writing equations is the most important part of the course. <br><br>		Future work will include better understanding why the displacement vector showed such a steep learning curve. At first, students were asking for lots of help and committing many mistakes. However, after making those initial attempts, they seemed to learn how to apply this knowledge component fairly quickly. We also plan to extend our analyses to include the equations that were written. From the student’s perspective, writing equations is the most important part of the course. <br><br>

Bobhaus: /* Physics concepts as knowledge components */

2009-05-14T17:17:49Z

Physics concepts as knowledge components

← Older revision		Revision as of 17:17, 14 May 2009
Line 57:		Line 57:

	===Physics concepts as knowledge components===		===Physics concepts as knowledge components===
	The analyses from the previous section suggest a closer examination of the vector learning curves. As the students move through the semester, they demonstrated slowly escalating assistance scores for drawing vectors. This is a very clear case where transfer is not occurring. Therefore, we decided to break down the vectors into their constituent physical concepts, which included drawing the acceleration, velocity, and displacement. The decomposed vector knowledge components are shown in Figure 5.<br>		The analyses from the previous section suggest a closer examination of the vector learning curves. As the students move through the semester, they demonstrated slowly escalating assistance scores for drawing vectors. This is a very clear case where transfer is not occurring. Therefore, we decided to break down the vectors into their constituent physical concepts, which included drawing the acceleration, velocity, and displacement. The decomposed vector knowledge components are shown in Figure 5.<br><br>
	According to the learning curves, it appears there is no transfer between drawing a translational displacement vector and drawing an angular displacement vector. At least initially, there is a huge jump between the first opportunity to apply this particular knowledge component (DRAW-DISPLACEMENT & DRAW-ANG-DISPLACEMENT), and then the assistance score returns to a low, asymptotic level. <br>		According to the learning curves, it appears there is no transfer between drawing a translational displacement vector and drawing an angular displacement vector. At least initially, there is a huge jump between the first opportunity to apply this particular knowledge component (DRAW-DISPLACEMENT & DRAW-ANG-DISPLACEMENT), and then the assistance score returns to a low, asymptotic level. <br><br>
	One potential explanation for the initial increase in assistance scores for displacement is in the way most rotational kinematics problems are worded. For example, the first problem in the USNA rotational homework set is, “A wheel is rotating counterclockwise at a constant rate of 3 rotations per second. Through what angle does the wheel rotate in 60.0 s?” It would be tempting for a novice to match the word “angle” in the problem statement, and use that as a basis for defining an angle in the Andes user interface. However, once the student attempts to define an angle, then the tutor will provide an unsolicited error message indicating that the angle is not part of the solution path for this problem. If the student then draws a displacement vector, then all of the errors and hints are blamed on the DRAW-ANG-DISPLACEMENT knowledge component (i.e., we use a temporal heuristic for the assignment of blame problem, Nwaigwe, Koedinger, VanLehn, Hausmann, & Weinstein, 2007).<br><br>		One potential explanation for the initial increase in assistance scores for displacement is in the way most rotational kinematics problems are worded. For example, the first problem in the USNA rotational homework set is, “A wheel is rotating counterclockwise at a constant rate of 3 rotations per second. Through what angle does the wheel rotate in 60.0 s?” It would be tempting for a novice to match the word “angle” in the problem statement, and use that as a basis for defining an angle in the Andes user interface. However, once the student attempts to define an angle, then the tutor will provide an unsolicited error message indicating that the angle is not part of the solution path for this problem. If the student then draws a displacement vector, then all of the errors and hints are blamed on the DRAW-ANG-DISPLACEMENT knowledge component (i.e., we use a temporal heuristic for the assignment of blame problem, Nwaigwe, Koedinger, VanLehn, Hausmann, & Weinstein, 2007).<br><br>

Bobhaus: /* The user-interface element as knowledge components */

2009-05-14T17:17:30Z

The user-interface element as knowledge components

← Older revision		Revision as of 17:17, 14 May 2009
Line 54:		Line 54:
	[[Image:INTERFACE.jpg\|Figure 4. A decomposition of each unit into knowledge components that correspond to the user interface.]]<br><br>		[[Image:INTERFACE.jpg\|Figure 4. A decomposition of each unit into knowledge components that correspond to the user interface.]]<br><br>

	Overall, there was a reliable difference between units, opportunities, and knowledge components, ~~‘‘F‘‘~~(26, 4352) = 24.56, ~~‘‘p‘‘~~ < .001. The overall effect was qualified by a three-way interaction, ~~‘‘F‘‘~~(8, 4352) = 2.82, ~~‘‘p‘‘~~ = .004. Using Figure 4 as a guide, we restricted our analyses to just the vector knowledge components as the students progressed through the curriculum. It appears that the amount of assistance needed to correctly apply a vector knowledge component grew with time. For the first opportunity, more assistance was needed to draw vectors in rotational kinematics than in the case of translational kinematics (~~‘‘p‘‘~~ < .001) and dynamics (~~‘‘p‘‘~~ < .001). The shape of the curves for the other two knowledge components was reasonable for the first opportunity.<br><br>		Overall, there was a reliable difference between units, opportunities, and knowledge components, ''F''(26, 4352) = 24.56, ''p'' < .001. The overall effect was qualified by a three-way interaction, ''F''(8, 4352) = 2.82, ''p'' = .004. Using Figure 4 as a guide, we restricted our analyses to just the vector knowledge components as the students progressed through the curriculum. It appears that the amount of assistance needed to correctly apply a vector knowledge component grew with time. For the first opportunity, more assistance was needed to draw vectors in rotational kinematics than in the case of translational kinematics (''p'' < .001) and dynamics (''p'' < .001). The shape of the curves for the other two knowledge components was reasonable for the first opportunity.<br><br>

	===Physics concepts as knowledge components===		===Physics concepts as knowledge components===