Reflective Dialogues (Katz) - Revision history

Connelly: /* Independent Variables */

2008-11-03T20:18:13Z

Independent Variables

← Older revision		Revision as of 20:18, 3 November 2008
Line 50:		Line 50:
	=== Independent Variables ===		=== Independent Variables ===

	Students within each course section were block-randomly assigned to one of two dialogue conditions: standard [[Knowledge Construction Dialogues]] (KCDs) or control with no KCDs. Students in the control condition were assigned an additional five Andes problems to attempt to better equate time on task.		Students within each course section were block-randomly assigned to one of two dialogue conditions: standard [[Knowledge Construction Dialogues]] (KCDs) or control with no KCDs. Control students worked on standard Andes problems only:
			[[Image:E1b-50.jpg]]

			KCD students worked through KCDs following selected Andes problems:
			[[Image:6kcd.jpg]]

			Students in the control condition were assigned an additional five Andes problems to attempt to better equate time on task.

	The following variables were entered into a regression analysis, with post-test score as the dependent variable:		The following variables were entered into a regression analysis, with post-test score as the dependent variable:

Connelly: /* Connections */

2008-04-15T21:28:37Z

Connections

← Older revision		Revision as of 21:28, 15 April 2008
Line 112:		Line 112:

	Use of Questions during learning		Use of Questions during learning
	* [[Post-practice reflection (Katz)\|Post-practice reflection (Katz & Connelly, ~~2006~~)]]		* [[Post-practice reflection (Katz)\|Post-practice reflection (Katz & Connelly, 2005)]]
	* [[Extending Reflective Dialogue Support (Katz & Connelly)\|Extending Reflective Dialogue Support (Katz & Connelly, ~~2008~~)]]		* [[Extending Reflective Dialogue Support (Katz & Connelly)\|Extending Reflective Dialogue Support (Katz & Connelly, 2007)]]
	* [[Craig_questions\|Deep-level questions during example studying (Craig & Chi)]]		* [[Craig_questions\|Deep-level questions during example studying (Craig & Chi)]]
	* [[FrenchCulture \| FrenchCulture (Amy Ogan, Christopher Jones, Vincent Aleven)]]		* [[FrenchCulture \| FrenchCulture (Amy Ogan, Christopher Jones, Vincent Aleven)]]

Connelly: /* Connections */

2008-04-15T21:24:23Z

Connections

← Older revision		Revision as of 21:24, 15 April 2008
Line 112:		Line 112:

	Use of Questions during learning		Use of Questions during learning
	* [[Post-practice reflection (Katz)]]		* [[Post-practice reflection (Katz)\|Post-practice reflection (Katz & Connelly, 2006)]]
			* [[Extending Reflective Dialogue Support (Katz & Connelly)\|Extending Reflective Dialogue Support (Katz & Connelly, 2008)]]
	* [[Craig_questions\|Deep-level questions during example studying (Craig & Chi)]]		* [[Craig_questions\|Deep-level questions during example studying (Craig & Chi)]]
	* [[FrenchCulture \| FrenchCulture (Amy Ogan, Christopher Jones, Vincent Aleven)]]		* [[FrenchCulture \| FrenchCulture (Amy Ogan, Christopher Jones, Vincent Aleven)]]

Connelly: /* Further Information */

2008-04-15T21:07:36Z

Further Information

← Older revision		Revision as of 21:07, 15 April 2008
Line 95:		Line 95:
	* Presentation to site visitors, 2005		* Presentation to site visitors, 2005
	* Full paper accepted at AIED 2007:		* Full paper accepted at AIED 2007:
	**Katz, S., Connelly, J., & Wilson, C. (~~in press~~). Out of the lab and into the classroom: An evaluation of reflective dialogue in Andes. ''~~Proceedings of AIED07~~''.		**Katz, S., Connelly, J., & Wilson, C. (2007). Out of the lab and into the classroom: An evaluation of reflective dialogue in Andes. In R. Luckin, K. R. Koedinger, & J. Greer (Eds.), ''Artificial Intelligence in Education: Building Technology Rich Learning Contexts that Work'' (pp. 425-432). Amsterdam: IOS Press.

	==== References ====		==== References ====

Connelly: /* Background and Significance */

2008-04-15T19:38:01Z

Background and Significance

← Older revision		Revision as of 19:38, 15 April 2008
Line 39:		Line 39:
	Because experts develop schemata largely through a good deal of practice with solving various types of problems, traditional introductory college physics courses take a similar approach. They are structured so that students are introduced to concepts, principles, and procedures in their text and through lectures, which they are then asked to apply to many problem-solving exercises. The hope is that, with extensive practice, students will integrate [[procedural]] and [[conceptual knowledge]] and develop expert-like schemata and planning skills. Unfortunately, many students don’t. In addition to exiting these courses with lingering naïve misconceptions (e.g., Halloun & Hestenes, 1985; McDermott, 1984), they continue to solve problems largely by manipulating equations until the desired quantity is isolated—that is, by means-end analysis—instead of by identifying relevant principles and generating solution plans before starting to solve a problem, as experts do (Dufresne et al., 1992; Larkin, McDermott, Simon, & Simon, 1980; Priest & Lindsay, 1992).		Because experts develop schemata largely through a good deal of practice with solving various types of problems, traditional introductory college physics courses take a similar approach. They are structured so that students are introduced to concepts, principles, and procedures in their text and through lectures, which they are then asked to apply to many problem-solving exercises. The hope is that, with extensive practice, students will integrate [[procedural]] and [[conceptual knowledge]] and develop expert-like schemata and planning skills. Unfortunately, many students don’t. In addition to exiting these courses with lingering naïve misconceptions (e.g., Halloun & Hestenes, 1985; McDermott, 1984), they continue to solve problems largely by manipulating equations until the desired quantity is isolated—that is, by means-end analysis—instead of by identifying relevant principles and generating solution plans before starting to solve a problem, as experts do (Dufresne et al., 1992; Larkin, McDermott, Simon, & Simon, 1980; Priest & Lindsay, 1992).

	We refer to the traditional approach to physics instruction described in the preceding paragraph as implicit instruction, because it encourages the inductive development of abstractions (concepts, principles, and schemata) through repeated exposure to instances, instead of by explicitly reifying these abstractions (O'Malley & Chamot, 1994). In response to the limitations of implicit approaches to physics instruction that neither ask students to plan nor scaffold them in doing it, several instructional scientists have proposed methods that explicitly engage students in planning exercises (Dufresne et al., 1992; Leonard, Dufresne, & Mestre, 1996; Mestre, Dufresne, Gerace, & Hardiman, 1993). These methods have met with modest success when tested mainly using high-achieving students (B or above in an introductory college physics course). We suggest that the main reason that many students, even high achievers, are unable to use explicit planning methods effectively is that they lack the basic concepts, principles, and procedures that are prerequisites for effective planning.		We refer to the traditional approach to physics instruction described in the preceding paragraph as [[implicit instruction]], because it encourages the inductive development of abstractions (concepts, principles, and schemata) through repeated exposure to instances, instead of by explicitly reifying these abstractions (O'Malley & Chamot, 1994). In response to the limitations of implicit approaches to physics instruction that neither ask students to plan nor scaffold them in doing it, several instructional scientists have proposed methods that explicitly engage students in planning exercises (Dufresne et al., 1992; Leonard, Dufresne, & Mestre, 1996; Mestre, Dufresne, Gerace, & Hardiman, 1993). These methods have met with modest success when tested mainly using high-achieving students (B or above in an introductory college physics course). We suggest that the main reason that many students, even high achievers, are unable to use explicit planning methods effectively is that they lack the basic concepts, principles, and procedures that are prerequisites for effective planning.

	This project tested the effectiveness of engaging students in reflective dialogues after they solve problems in Andes (e.g., Gertner & VanLehn, 2000) which explicitly targeted the three main types of knowledge that experts employ during planning: knowledge about what principle(s) to apply to a given problem, how to apply these principles (e.g., what equations to use), and why to apply them—that is, what the applicability conditions are (Leonard, Dufresne, & Mestre, 1996). Explicit instruction on these knowledge components proved to be effective and more efficient than giving students repeated practice in solving different types of problems.		This project tested the effectiveness of engaging students in reflective dialogues after they solve problems in Andes (e.g., Gertner & VanLehn, 2000) which explicitly targeted the three main types of knowledge that experts employ during planning: knowledge about what principle(s) to apply to a given problem, how to apply these principles (e.g., what equations to use), and why to apply them—that is, what the applicability conditions are (Leonard, Dufresne, & Mestre, 1996). Explicit instruction on these knowledge components proved to be effective and more efficient than giving students repeated practice in solving different types of problems.

Connelly: /* Findings */

2008-04-10T20:25:01Z

Findings

← Older revision		Revision as of 20:25, 10 April 2008
Line 64:		Line 64:

	=== Findings ===		=== Findings ===
	~~Data analyses are still in progress, but below are our findings to date:~~

	Analyses of pre- and post-test scores were more encouraging than for last year's study ([[Post-practice reflection (Katz)]]). After omitting scores from one student with a post-test duration of less than two minutes, we were left with treatment and control groups of equal size (''n'' = 33). We also re-classified two treatment subjects who did no dialogues as control subjects (effective treatment and control ''n''s = 31 and 35, respectively), although these subjects were not able to access the five extra control-group Andes problems. As we had hoped, ANOVA showed no significant differences between the effective treatment and control groups on pre-test score (respective ''M''s = 12.10 and 11.77; ''F'' < 1), but treatment subjects had higher mean post-test scores (17.97 vs. 15.57; ''F''(1, 64) = 4.89, ''p'' = .031), mean raw gain scores (5.87 vs. 3.80; ''F'' = 5.62, ''p'' = .021), and mean Estes gain scores (0.330 vs. 0.208; ''F'' = 6.74, ''p'' = .012) than did control subjects. In short, without regard to problem-solving practice, subjects who did KCDs did significantly better on the post-test than those who did not.		Analyses of pre- and post-test scores were more encouraging than for last year's study ([[Post-practice reflection (Katz)]]). After omitting scores from one student with a post-test duration of less than two minutes, we were left with treatment and control groups of equal size (''n'' = 33). We also re-classified two treatment subjects who did no dialogues as control subjects (effective treatment and control ''n''s = 31 and 35, respectively), although these subjects were not able to access the five extra control-group Andes problems. As we had hoped, ANOVA showed no significant differences between the effective treatment and control groups on pre-test score (respective ''M''s = 12.10 and 11.77; ''F'' < 1), but treatment subjects had higher mean post-test scores (17.97 vs. 15.57; ''F''(1, 64) = 4.89, ''p'' = .031), mean raw gain scores (5.87 vs. 3.80; ''F'' = 5.62, ''p'' = .021), and mean Estes gain scores (0.330 vs. 0.208; ''F'' = 6.74, ''p'' = .012) than did control subjects. In short, without regard to problem-solving practice, subjects who did KCDs did significantly better on the post-test than those who did not.

Connelly: /* Background and Significance */

2008-04-10T20:23:56Z

Background and Significance

← Older revision		Revision as of 20:23, 10 April 2008
Line 37:		Line 37:

	=== Background and Significance ===		=== Background and Significance ===
	Because experts develop schemata largely through a good deal of practice with solving various types of problems, traditional introductory college physics courses take a similar approach. They are structured so that students are introduced to concepts, principles, and procedures in their text and through lectures, which they are then asked to apply to many problem-solving exercises. The hope is that, with extensive practice, students will integrate ~~conceptual and~~ [[procedural]] knowledge and develop expert-like schemata and planning skills. Unfortunately, many students don’t. In addition to exiting these courses with lingering naïve misconceptions (e.g., Halloun & Hestenes, 1985; McDermott, 1984), they continue to solve problems largely by manipulating equations until the desired quantity is isolated—that is, by means-end analysis—instead of by identifying relevant principles and generating solution plans before starting to solve a problem, as experts do (Dufresne et al., 1992; Larkin, McDermott, Simon, & Simon, 1980; Priest & Lindsay, 1992).		Because experts develop schemata largely through a good deal of practice with solving various types of problems, traditional introductory college physics courses take a similar approach. They are structured so that students are introduced to concepts, principles, and procedures in their text and through lectures, which they are then asked to apply to many problem-solving exercises. The hope is that, with extensive practice, students will integrate [[procedural]] and [[conceptual knowledge]] and develop expert-like schemata and planning skills. Unfortunately, many students don’t. In addition to exiting these courses with lingering naïve misconceptions (e.g., Halloun & Hestenes, 1985; McDermott, 1984), they continue to solve problems largely by manipulating equations until the desired quantity is isolated—that is, by means-end analysis—instead of by identifying relevant principles and generating solution plans before starting to solve a problem, as experts do (Dufresne et al., 1992; Larkin, McDermott, Simon, & Simon, 1980; Priest & Lindsay, 1992).

	We refer to the traditional approach to physics instruction described in the preceding paragraph as implicit instruction, because it encourages the inductive development of abstractions (concepts, principles, and schemata) through repeated exposure to instances, instead of by explicitly reifying these abstractions (O'Malley & Chamot, 1994). In response to the limitations of implicit approaches to physics instruction that neither ask students to plan nor scaffold them in doing it, several instructional scientists have proposed methods that explicitly engage students in planning exercises (Dufresne et al., 1992; Leonard, Dufresne, & Mestre, 1996; Mestre, Dufresne, Gerace, & Hardiman, 1993). These methods have met with modest success when tested mainly using high-achieving students (B or above in an introductory college physics course). We suggest that the main reason that many students, even high achievers, are unable to use explicit planning methods effectively is that they lack the basic concepts, principles, and procedures that are prerequisites for effective planning.		We refer to the traditional approach to physics instruction described in the preceding paragraph as implicit instruction, because it encourages the inductive development of abstractions (concepts, principles, and schemata) through repeated exposure to instances, instead of by explicitly reifying these abstractions (O'Malley & Chamot, 1994). In response to the limitations of implicit approaches to physics instruction that neither ask students to plan nor scaffold them in doing it, several instructional scientists have proposed methods that explicitly engage students in planning exercises (Dufresne et al., 1992; Leonard, Dufresne, & Mestre, 1996; Mestre, Dufresne, Gerace, & Hardiman, 1993). These methods have met with modest success when tested mainly using high-achieving students (B or above in an introductory college physics course). We suggest that the main reason that many students, even high achievers, are unable to use explicit planning methods effectively is that they lack the basic concepts, principles, and procedures that are prerequisites for effective planning.

Katz: /* Future plans */

2008-01-10T04:03:30Z

Future plans

← Older revision		Revision as of 04:03, 10 January 2008
Line 125:		Line 125:

	==== Future plans ====		==== Future plans ====
	Our future plans for ~~June 2007~~ - December ~~2007~~:		Our future plans for January 2008 - December 2008:
	* write journal article expanding AIED conference paper		* write journal article expanding AIED conference paper

	[[Category:Study]]		[[Category:Study]]

Katz: /* Findings */

2008-01-10T04:01:15Z

Findings

← Older revision		Revision as of 04:01, 10 January 2008
Line 78:		Line 78:

	In summary, student performance on the post-test relative to the pre-test was significantly influenced by the number of dialogues they completed, as opposed to the number of target problems they completed prior to the post-test. However, neither measure had a significant effect on student performance on an exam that focused on		In summary, student performance on the post-test relative to the pre-test was significantly influenced by the number of dialogues they completed, as opposed to the number of target problems they completed prior to the post-test. However, neither measure had a significant effect on student performance on an exam that focused on
	problem-solving ability~~. Analyses of the final exam data are in progress~~.		problem-solving ability.

	=== Explanation ===		=== Explanation ===

Katz: /* Hypothesis */

2008-01-10T03:55:56Z

Hypothesis

← Older revision		Revision as of 03:55, 10 January 2008
Line 61:		Line 61:

	=== Hypothesis ===		=== Hypothesis ===
	The main hypothesis ~~to be~~ tested is that explicit, part-task training on the “what? how? and why?” components of planning, via strategically staged reflective dialogues, ~~will~~ be more effective and efficient than the traditional approach of letting students acquire these components implicitly, through lots of practice with solving problems.		The main hypothesis tested was that explicit, part-task training on the “what? how? and why?” components of planning, via strategically staged reflective dialogues, would be more effective and efficient than the traditional approach of letting students acquire these components implicitly, through lots of practice with solving problems.

	=== Findings ===		=== Findings ===

← Older revision		Revision as of 21:28, 15 April 2008
Line 112:		Line 112:

	Use of Questions during learning		Use of Questions during learning
	* [[Post-practice reflection (Katz)\|Post-practice reflection (Katz & Connelly, ~~2006~~)]]		* [[Post-practice reflection (Katz)\|Post-practice reflection (Katz & Connelly, 2005)]]
	* [[Extending Reflective Dialogue Support (Katz & Connelly)\|Extending Reflective Dialogue Support (Katz & Connelly, ~~2008~~)]]		* [[Extending Reflective Dialogue Support (Katz & Connelly)\|Extending Reflective Dialogue Support (Katz & Connelly, 2007)]]
	* [[Craig_questions\|Deep-level questions during example studying (Craig & Chi)]]		* [[Craig_questions\|Deep-level questions during example studying (Craig & Chi)]]
	* [[FrenchCulture \| FrenchCulture (Amy Ogan, Christopher Jones, Vincent Aleven)]]		* [[FrenchCulture \| FrenchCulture (Amy Ogan, Christopher Jones, Vincent Aleven)]]