Difference between revisions of "Note-Taking Technologies"

From LearnLab
Jump to: navigation, search
(Background/Significance)
 
(33 intermediate revisions by 2 users not shown)
Line 5: Line 5:
 
Via several completed and proposed experiments, we are exploring two general hypothesis regarding the effect of note-taking on learning. First, we believe that note-taking encourages active processing, and thus long-term retention, when it requires students to attend to the critical elements of the learning material. This results in increased [[feature validity]] of the mental representation of the [[knowledge component]]. This falls within the [[Refinement and Fluency]] cluster. Our second hypothesis is that note-taking facilitates [[long-term retention]] when it involves the use of multiple representations of concepts. This falls within the [[Coordinative Learning]] cluster. Our studies are designed to evaluate these hypotheses by comparing note-taking tools offering different functionality.  
 
Via several completed and proposed experiments, we are exploring two general hypothesis regarding the effect of note-taking on learning. First, we believe that note-taking encourages active processing, and thus long-term retention, when it requires students to attend to the critical elements of the learning material. This results in increased [[feature validity]] of the mental representation of the [[knowledge component]]. This falls within the [[Refinement and Fluency]] cluster. Our second hypothesis is that note-taking facilitates [[long-term retention]] when it involves the use of multiple representations of concepts. This falls within the [[Coordinative Learning]] cluster. Our studies are designed to evaluate these hypotheses by comparing note-taking tools offering different functionality.  
  
 +
View the [[#Descendents|'Descendents']] section to view experiments associated with these hypotheses.
  
 
== Glossary ==
 
== Glossary ==
  
''Select/ion:'' This term is used in the context of this study to identify the behavior of using the mouse and cursor to actively highlight a portion of digital text. Selection is first step for several online note-taking techniques, including copy-paste and annotation.
+
''Selection:'' This term is used in the context of this study to identify the behavior of using the mouse and cursor to actively highlight a portion of digital text. Selection is first step for several online note-taking techniques, including copy-paste and annotation.
  
 
''Copy-Paste:'' This is the act of selecting material, copying it to the computer clipboard (via a keyboard shortcut or menu), and then pasting it into students’ notes.  
 
''Copy-Paste:'' This is the act of selecting material, copying it to the computer clipboard (via a keyboard shortcut or menu), and then pasting it into students’ notes.  
Line 16: Line 17:
 
== Research Question ==
 
== Research Question ==
  
Can we improve learning outcomes in online courses by changing the way students can take notes?  
+
Practical question: Can we improve learning outcomes in online courses by changing the way students can take notes?
  
== Background/Significance ==
 
  
In a previous study, we showed that copy-paste functionality in a note-taking application appears to reduce [[long-term retention]] relative to note-taking using typing or handwriting. This study evaluates two hypotheses regarding this effect. In the previous study we found that when using copy-paste functionality, ideas that were recorded in a more wordy fashion were forgotten more than less wordy notes. This was not the case for typed or handwritten notes. As the cost of wordy notes is low using copy-paste, we believed that wordy notes are indicative of decreased attention to the idea being recorded. In addition, while in typing and handwriting wordiness is associated with more time spent on the idea being recorded, this is not the case for copy-paste. We designed a tool to restrict the amount of text students could select in any one copy-paste action in order to increase the attention required to record a note. We believed this would result in superior learning. It is important to note that whereas in our previous study all tools had typing functionality, in this case only users of the typing tool could type.
+
Scientific question: What are the cognitive mechanisms underlying learning gains from note-taking?
  
While students create their own notes through typing or handwriting, they simply copy material verbatim using copy-paste functionality. It may be that rewording of notes is important, or it may be that simply viewing an alternative representation of the learning material in notes is key to learning gains. To tease these apart, we designed a selection tool that, upon a student selecting an idea to record, gave the students three options to place in their notes. One was a reworded version of the idea recorded, and the other two were distractors. If it was only important to view an alternative wording, this tool should show superior performance to copy-paste. If involvement in the rewording was important, the tool should show equivalent performance to copy-paste.
+
== Background/Significance ==
  
== Dependent Variables ==
+
Note-taking research has shown that the process of taking notes can have a positive impact on long-term retention (Kobayashi 2005, 2006). Two main processes have been proposed to explain the mechanisms behind learning gains. The attention hypothesis is that "note-taking forces the learner to pay more attention to the presented material", while the generation hypothesis states that note-taking causes students to "actively relate the material to existing knowledge" (Peper and Mayer, 1986). There is little behavioral data to support either of these hypothesis. Technology may allows us to investigate these hypothesis in greater detail, as our preliminary studies indicate it provides a level of control over the note-taking process, and thus behavior.
  
'''Tests'''
+
It is useful to cast the two above hypotheses in PSLC terms. The attention hypothesis can be restated with regards to the [[focusing]] aspect of [[Refinement and Fluency]]. Note-taking thus facilitates learning when it requires students to focus on the critical [[knowledge components]] of the learning material. The generation hypothesis can be restated with regards to the [[Coordinative Learning]] cluster. Note-taking facilitates learning when it requires students to coordinate multiple versions of the same learning material.
  
[[Note:]] all tests include both multiple choice and free response questions. The multiple choice questions all involve solving problems (for example, given a response structure, which variables are direct causes of an effect, or which interact to produce an effect). In addition, some free response questions ask students to explain terminology used in the module.
+
Our [[Note-Taking:_Restriction_and_Selection|preliminary results]] indicate some support for the [[focusing]] hypothesis. We found that when students use copy-paste functionality, they performed worse on items that they recorded in a wordy fashion than they did on items they recorded more efficiently. The wordiness may be an indication of students' [[focusing|focus]], showing a lack of attention to the critical components of the ideas. In addition, copy-paste functionality increased the number of items students recorded when compared to note-taking via typing. This may be an indication of decreased attention to key ideas.  
  
''Normal Learning, immediate:'' Students are given a test immediately after studying the material.
+
Technology offers a valuable comparison by which to evaluate the coordination hypothesis. While copy-paste note-taking involves the creation of a secondary notepad, which is available at all times, highlighting does not involve a notepad. Coordinating the information in the notepad with the information in the window may promote learning, much as seen in Wiley's work (Wiley, 2001).
  
''Long-Term Retention, Normal Learning:'' Students return a week following the treatment (which lasts between 30 and 90 minutes) to take this test.
+
== Dependent Variables ==
 
+
Described on individual study pages. [[Long-term retention]] measures are included in all but the initial study.
''Normal Learning, review:'' After taking the long-term retention test, students are given their notes to review for 5 minutes. Following this review period, students take a final test.
 
 
 
 
 
'''Behavior'''
 
 
 
''Note-Quantity:'' The total number of ideas students place in their notes is captured, as well as the number of words used to express those ideas.
 
 
 
''Note-Wording:'' How students word their notes is recorded. Each ideas is either recorded Verbatim, Abbreviated, or in students Own words.
 
 
 
''Completion Time:'' The time students take to complete the learning material is recorded.
 
 
 
'''Motivation/Interest'''
 
 
 
''Experience:'' After taking the final test, students are given a survey which solicits their reaction to the tool they used. They are asked to identify their most and least favorite features of the tools, and how they believe the tool affected their note-taking behavior.
 
  
 
== Independent Variables ==
 
== Independent Variables ==
 +
[[image:NT-TextEditor.gif]]
  
'''Note-taking Treatment'''
+
''This is the basic note-taking text-editor. Built in javascript, the editor occupies the lower third of the screen, while the learning materials occupy the top of the screen. The functionality of the various manipulations is described on the individual study pages.
 
 
''Paste:'' Students can only create notes by copy-pasting material from the learning content to their notepad. Students can select as much material as they like in any single pasting action.
 
 
 
''Restricted Paste:'' Students can only create notes by copy-pasting material from the learning content to their notepad. Students are restricted to selecting 90% of the words of any single sentence they are selecting.
 
 
 
''Typing:'' Students can only create notes by typing directly into their notepad.
 
 
 
''Selection-Tool:'' Students can only create notes by choosing one of the 3 options made available when they select learning material.
 
<blockquote>'''Image of the Selection Tool'''<br>[[image:Note-SelectTool.gif|200]]</blockquote>
 
 
 
''Highlighting:'' Students can create notes by actively highlighting course material. These highlights are seen with differently colored backgrounds, as would be the case with a physical highlighter.
 
 
 
'''Context/Mediating Variables on Student Characteristics'''
 
 
 
''SAT Score:'' All students are asked to provide their SAT scores, as in previous studies SAT-Math was found to be an important covariate.
 
 
 
''Pretest score:'' Prior to the learning material, students take a pre-test similar to the normal tests described above.
 
 
 
''Preferences:'' In the survey, students are asked how they prefer to take notes in their regular student-life.
 
  
 
== Hypotheses ==
 
== Hypotheses ==
1. 'Refinement': Restrictions on selections will increase the students' [[focusing|focus]] on the ideas being recorded, resulting in improved learning gains relative to unrestricted copy-paste.
+
1. ''Attention/Fluency-[[Refinement]]:''  Note-taking benefits students when it requires them to focus on the critical components of the ideas they are recording. Restricting the amount of material students can select in any individual selection behavior will increase the attention paid to critical features of the learning material. This will result in improved retention compared to unrestricted selection.  
  
2. 'Coordination': Viewing multiple versions of the same idea will require students to coordinate material, resulting in improved learning gains (less forgetting) relative to copy-paste.
+
2. ''Coordinative Learning/Additional Representation:'' Note-taking benefits students because it allows them to simultaneously [[coordinate]] two representations of the same material, the fixed one created by the content author, and their own set of notes.
  
 
== Findings ==
 
== Findings ==
[[Image:NT-Exp3-Learn.gif]]
+
* Copy-Paste vs. Typing
 
+
** [[Note-Taking:_Copy_Paste|Copy-paste, when combined with Typing functionality, appears to result in reduced long-term retention than Typing alone. ]]
*''Learning:'' The two novel tools (bottom) showed significantly lower learning on all tests than did the standard tools(top two). The two novel tools (Selection and Restricted) were not significantly different from each other. The two standard tools (Copy-Paste and Typing) were not significantly different from each other). There was a significant overall effect for review (tests 2-3) across all conditions, but no significant effect for forgetting (tests 1 to 2).
+
** [[Note-Taking:_Restriction_and_Selection|Copy-paste, when alone, appears to result in more efficient learning than typing alone. Students learn the same amount in less time.]]
 
+
** [[Note-Taking:_Copy_Paste|Given both copy-paste and typing functionality, students' will tend to paste ideas more often then they type ideas.]]
[[Image:NT-Exp3-Efficiency.gif]]
+
* [[Note-Taking:_Restriction_and_Selection|When students record wordy ideas using copy-paste functionality, they perform worse on learning outcomes than when they record ideas using fewer words.]]
 
+
* [[Note-Taking:_Restriction_and_Selection|Students have a certain resistance to novel interaction techniques that makes it difficult to create interventions to evaluate note-taking hypothesis. This indicates the need for careful design of interventions.]]
*''Efficiency:'' The Paste condition showed more efficient learning than all other tools on the immediate test (1) and review test (3). Efficiency is defined as learning difference between two subsequent tests divided by the time spend studying the materials.
+
** [[Note-Taking:_Restriction_and_Selection|These novel tools produce inferior learning to standard interaction techniques.]]
 
 
[[Image:NT-Exp3-Ideas.gif]]
 
 
 
*''Ideas Recorded:'' The Paste tool recorded significantly more total ideas than any other condition, and the same amount of key ideas as the typing condition. The Novel tools recorded significantly fewer key ideas than the standard conditions.
 
*''Wordiness:'' In line with previous experiments, when students recorded wordier ideas using copy-paste functionality, they performed worse on test items associated with the idea than when they recorded less wordy ideas.
 
*''Desirability:'' Students were significantly more dissatisfied with the selection tool than they were with any other tool. Most who used the restricted tool reported disliking the restrictions.
 
  
 
== Explanation ==
 
== Explanation ==
*The results of this experiment do not support our hyptheses, as the novel tools performed worse on learning outcomes than did the standard tools. However, this may be due to usability issues. The Selection tool was the most disliked tool, and users reported disliking the restrictions in the restrited-paste tool. In addition, note-taking behavior was suboptimal, as the novel conditions recorded fewer key ideas than did the standard tools. It may be that the design of the novel tools required more attention to the process of note-taking, rather than the product.
+
See individual experiment pages listed below in the Descendents section.
**Lesson: More attention must be paid to the design of note-taking interventions, in order to ensure adoption by users.
 
*When students are only allowed to copy-paste, they appear to learn as much as when they type. Contrary to previous results, they do not forget more over time. In contrast to the previous experiment, students in the copy-paste condition could not type. This may have increased their attention to the process of pasting. In addition, students in the copy-paste condition showed reduced time on task compared to typing.
 
**Lesson: Copy-Paste may be a more efficient note-taking technique than typing.
 
  
 
== Descendents ==
 
== Descendents ==
 
+
'''Completed Experiments'''
[[Note-Taking: Coordination]]
+
*[[Note-Taking: Copy Paste]]: compares Paste and Type with Typing-Only and Pencil-and-Paper
 +
*[[Note-Taking: Restriction and Selection]]: compares Typing, Paste-Only, Restricted-Paste, and Selection
 +
'''Planned Experiments'''
 +
*[[Note-Taking: Coordination]]: Evaluates the hypothesis that positive note-taking involves coordinating the notepad with the learning materials by comparing performance using copy-paste with performance using highlighting.
 +
*[[Note-Taking: Focusing On Concepts]]: Evaluates the hypothesis that positive note-taking involves attention to what is being recorded by comparing unrestricted copy-paste with a restricted copy-paste.
 +
*[[Note-Taking: Focusing On Quantity]]: Evaluates the hypothesis that positive note-taking involves [[focusing]] on key concepts by comparing a tool that allows students to record as many ideas as they desire with a tool that limits the number of notes students can record.
  
 
== Further Information ==
 
== Further Information ==
 +
'''References'''
 +
# Kobayashi, K. (2005). What Limits the Encoding Effect of Note-Taking? A meta-analytic examination., Contemporary Educational Psychology, 30, 242-262
 +
# Kobayashi, K. (2006). Combined Effects of Note-Taking/-Reviewing on Learning and the Enhancement Through Interventions: A meta-analytic review. (1986). Educational Psychology 26, 3 (2006) 459-477
 +
# Peper, R.J., Mayer, R.E., Generative Effects of Note taking During Science Lectures. Journal of Educational Psychology 78, 1  34-38
 +
#Wiley, J. (2001) Supporting understanding through task and browser design. Proceedings of the Twenty-third annual Conference of the Cognitive Science Society, (pp. 1136-1143). Hillsdale, NJ: Erlbaum.
  
The following is a reference to a conference paper regarding our earlier study contrasting handwriting, typing, and copy-paste.
+
The following papers report our earlier studies contrasting handwriting, typing, and copy-paste.
  
Bauer, A., Koedinger, K. Pasting and Encoding: Note-taking in Online Courses. IEEE International Conference on Advanced Learning Technologies (ICALT 2006), 5-7 July, Kerkrade, Netherlands.
+
# Bauer, A., Koedinger, K. Pasting and Encoding: Note-taking in Online Courses. IEEE International Conference on Advanced Learning Technologies (ICALT 2006), 5-7 July, Kerkrade, Netherlands.
  
- Bauer, A., Koedinger. K.R., Selection-Based Note-Taking Applications, ACM Symposium on Human Factors in Computing Systems 2007, in submission
+
# Bauer, A., Koedinger. K.R., Selection-Based Note-Taking Applications, ACM Symposium on Human Factors in Computing Systems 2007, accepted for publication

Latest revision as of 23:49, 14 April 2007

Abstract

We are conducting an investigation into the relationship between note-taking and learning in online courses. The literature has shown that the process of taking notes can have a positive impact on long-term retention. Our completed studies indicate that the features included in online note-taking applications can have an effect on these process benefits. Analyses of our results have led us to explore the effect of selection-based note-taking on both behavior and learning.

Via several completed and proposed experiments, we are exploring two general hypothesis regarding the effect of note-taking on learning. First, we believe that note-taking encourages active processing, and thus long-term retention, when it requires students to attend to the critical elements of the learning material. This results in increased feature validity of the mental representation of the knowledge component. This falls within the Refinement and Fluency cluster. Our second hypothesis is that note-taking facilitates long-term retention when it involves the use of multiple representations of concepts. This falls within the Coordinative Learning cluster. Our studies are designed to evaluate these hypotheses by comparing note-taking tools offering different functionality.

View the 'Descendents' section to view experiments associated with these hypotheses.

Glossary

Selection: This term is used in the context of this study to identify the behavior of using the mouse and cursor to actively highlight a portion of digital text. Selection is first step for several online note-taking techniques, including copy-paste and annotation.

Copy-Paste: This is the act of selecting material, copying it to the computer clipboard (via a keyboard shortcut or menu), and then pasting it into students’ notes.

Highlighting: The act of creating a lasting distinction between the selected text and the main content. For example, through creating a yellow background or underlining the text.

Research Question

Practical question: Can we improve learning outcomes in online courses by changing the way students can take notes?


Scientific question: What are the cognitive mechanisms underlying learning gains from note-taking?

Background/Significance

Note-taking research has shown that the process of taking notes can have a positive impact on long-term retention (Kobayashi 2005, 2006). Two main processes have been proposed to explain the mechanisms behind learning gains. The attention hypothesis is that "note-taking forces the learner to pay more attention to the presented material", while the generation hypothesis states that note-taking causes students to "actively relate the material to existing knowledge" (Peper and Mayer, 1986). There is little behavioral data to support either of these hypothesis. Technology may allows us to investigate these hypothesis in greater detail, as our preliminary studies indicate it provides a level of control over the note-taking process, and thus behavior.

It is useful to cast the two above hypotheses in PSLC terms. The attention hypothesis can be restated with regards to the focusing aspect of Refinement and Fluency. Note-taking thus facilitates learning when it requires students to focus on the critical knowledge components of the learning material. The generation hypothesis can be restated with regards to the Coordinative Learning cluster. Note-taking facilitates learning when it requires students to coordinate multiple versions of the same learning material.

Our preliminary results indicate some support for the focusing hypothesis. We found that when students use copy-paste functionality, they performed worse on items that they recorded in a wordy fashion than they did on items they recorded more efficiently. The wordiness may be an indication of students' focus, showing a lack of attention to the critical components of the ideas. In addition, copy-paste functionality increased the number of items students recorded when compared to note-taking via typing. This may be an indication of decreased attention to key ideas.

Technology offers a valuable comparison by which to evaluate the coordination hypothesis. While copy-paste note-taking involves the creation of a secondary notepad, which is available at all times, highlighting does not involve a notepad. Coordinating the information in the notepad with the information in the window may promote learning, much as seen in Wiley's work (Wiley, 2001).

Dependent Variables

Described on individual study pages. Long-term retention measures are included in all but the initial study.

Independent Variables

NT-TextEditor.gif

This is the basic note-taking text-editor. Built in javascript, the editor occupies the lower third of the screen, while the learning materials occupy the top of the screen. The functionality of the various manipulations is described on the individual study pages.

Hypotheses

1. Attention/Fluency-Refinement: Note-taking benefits students when it requires them to focus on the critical components of the ideas they are recording. Restricting the amount of material students can select in any individual selection behavior will increase the attention paid to critical features of the learning material. This will result in improved retention compared to unrestricted selection.

2. Coordinative Learning/Additional Representation: Note-taking benefits students because it allows them to simultaneously coordinate two representations of the same material, the fixed one created by the content author, and their own set of notes.

Findings

Explanation

See individual experiment pages listed below in the Descendents section.

Descendents

Completed Experiments

Planned Experiments

  • Note-Taking: Coordination: Evaluates the hypothesis that positive note-taking involves coordinating the notepad with the learning materials by comparing performance using copy-paste with performance using highlighting.
  • Note-Taking: Focusing On Concepts: Evaluates the hypothesis that positive note-taking involves attention to what is being recorded by comparing unrestricted copy-paste with a restricted copy-paste.
  • Note-Taking: Focusing On Quantity: Evaluates the hypothesis that positive note-taking involves focusing on key concepts by comparing a tool that allows students to record as many ideas as they desire with a tool that limits the number of notes students can record.

Further Information

References

  1. Kobayashi, K. (2005). What Limits the Encoding Effect of Note-Taking? A meta-analytic examination., Contemporary Educational Psychology, 30, 242-262
  2. Kobayashi, K. (2006). Combined Effects of Note-Taking/-Reviewing on Learning and the Enhancement Through Interventions: A meta-analytic review. (1986). Educational Psychology 26, 3 (2006) 459-477
  3. Peper, R.J., Mayer, R.E., Generative Effects of Note taking During Science Lectures. Journal of Educational Psychology 78, 1 34-38
  4. Wiley, J. (2001) Supporting understanding through task and browser design. Proceedings of the Twenty-third annual Conference of the Cognitive Science Society, (pp. 1136-1143). Hillsdale, NJ: Erlbaum.

The following papers report our earlier studies contrasting handwriting, typing, and copy-paste.

  1. Bauer, A., Koedinger, K. Pasting and Encoding: Note-taking in Online Courses. IEEE International Conference on Advanced Learning Technologies (ICALT 2006), 5-7 July, Kerkrade, Netherlands.
  1. Bauer, A., Koedinger. K.R., Selection-Based Note-Taking Applications, ACM Symposium on Human Factors in Computing Systems 2007, accepted for publication