Note-Taking: Restriction and Selection

Abstract

This study evaluates two hypothesis regarding negative learning effects observed for note-taking tools with copy-paste functionality. Two novel tools are compared with two tools that use standard functionality: one that only allows copy-paste, and the other that only allows typing. The first novel tool restricts the amount of text students can select in any one copy-paste action, in order to increase attention to what is being recorded. The second requires students to choose a reworded version of the idea selected, in order to evaluate whether it is important that students create their notes or simply view reworded versions of the ideas they want to record. Our study found that both novel tools performed worse on learning outcomes than the standard tools, which may be due to an observed user frustration with the novel interaction techniques. We also found that copy-paste only functionality increased the efficiency of learning relative to typing. In other words, people using the copy-paste tool learned the same amount in less time than users of the typing tool.

Glossary

Select/ion: This term is used in the context of this study to identify the behavior of using the mouse and cursor to actively highlight a portion of digital text. Selection is first step for several online note-taking techniques, including copy-paste and annotation.

Copy-Paste: This is the act of selecting material, copying it to the computer clipboard (via a keyboard shortcut or menu), and then pasting it into students’ notes.

Highlighting: The act of creating a lasting distinction between the selected text and the main content. For example, through creating a yellow background or underlining the text.

Research Question

Can we improve learning outcomes in online courses by changing the way students can take notes?

Background/Significance

In a previous study, we showed that copy-paste functionality in a note-taking application appears to reduce long-term retention relative to note-taking using typing or handwriting. This study evaluates two hypotheses regarding this effect. In the previous study we found that when using copy-paste functionality, ideas that were recorded in a more wordy fashion were forgotten more than less wordy notes. This was not the case for typed or handwritten notes. As the cost of wordy notes is low using copy-paste, we believed that wordy notes are indicative of decreased attention to the idea being recorded. In addition, while in typing and handwriting wordiness is associated with more time spent on the idea being recorded, this is not the case for copy-paste. We designed a tool to restrict the amount of text students could select in any one copy-paste action in order to increase the attention required to record a note. We believed this would result in superior learning. It is important to note that whereas in our previous study all tools had typing functionality, in this case only users of the typing tool could type.

While students create their own notes through typing or handwriting, they simply copy material verbatim using copy-paste functionality. It may be that rewording of notes is important, or it may be that simply viewing an alternative representation of the learning material in notes is key to learning gains. To tease these apart, we designed a selection tool that, upon a student selecting an idea to record, gave the students three options to place in their notes. One was a reworded version of the idea recorded, and the other two were distractors. If it was only important to view an alternative wording, this tool should show superior performance to copy-paste. If involvement in the rewording was important, the tool should show equivalent performance to copy-paste.

Dependent Variables

Tests

Note: all tests include both multiple choice and free response questions. The multiple choice questions all involve solving problems (for example, given a response structure, which variables are direct causes of an effect, or which interact to produce an effect). In addition, some free response questions ask students to explain terminology used in the module.

Normal Learning, immediate: Students are given a test immediately after studying the material.

Long-Term Retention, Normal Learning: Students return a week following the treatment (which lasts between 30 and 90 minutes) to take this test.

Normal Learning, review: After taking the long-term retention test, students are given their notes to review for 5 minutes. Following this review period, students take a final test.

Behavior

Note-Quantity: The total number of ideas students place in their notes is captured, as well as the number of words used to express those ideas.

Note-Wording: How students word their notes is recorded. Each ideas is either recorded Verbatim, Abbreviated, or in students Own words.

Completion Time: The time students take to complete the learning material is recorded.

Motivation/Interest

Experience: After taking the final test, students are given a survey which solicits their reaction to the tool they used. They are asked to identify their most and least favorite features of the tools, and how they believe the tool affected their note-taking behavior.

Independent Variables

Note-taking Treatment

Paste: Students can only create notes by copy-pasting material from the learning content to their notepad. Students can select as much material as they like in any single pasting action.

Restricted Paste: Students can only create notes by copy-pasting material from the learning content to their notepad. Students are restricted to selecting 90% of the words of any single sentence they are selecting.

Typing: Students can only create notes by typing directly into their notepad.

Selection-Tool: Students can only create notes by choosing one of the 3 options made available when they select learning material.

Image of the Selection Tool

Highlighting: Students can create notes by actively highlighting course material. These highlights are seen with differently colored backgrounds, as would be the case with a physical highlighter.

Context/Mediating Variables on Student Characteristics

SAT Score: All students are asked to provide their SAT scores, as in previous studies SAT-Math was found to be an important covariate.

Pretest score: Prior to the learning material, students take a pre-test similar to the normal tests described above.

Preferences: In the survey, students are asked how they prefer to take notes in their regular student-life.

Hypotheses

1. 'Refinement': Restrictions on selections will increase the students' focus on the ideas being recorded, resulting in improved learning gains relative to unrestricted copy-paste.

2. 'Coordination': Viewing multiple versions of the same idea will require students to coordinate material, resulting in improved learning gains (less forgetting) relative to copy-paste.

Findings

- Copy-paste takes less time to produce the same learning results as Typing. Though copy-paste trends towards forgetting more at a delay than Typing, this effect was only significant in our first study.

- Students who copy-paste do better when they take less wordy notes. This is not the case for other tools. This provides some evidence for the attention hypothesis.

- Usability issues impaired our ability to directly address the attention and coordination hypotheses. Students disliked both the Selection and the Restricted-Paste tool, and used them less than the more standard Typing and Copy-Paste tools.

Explanation

The current study was ineffective in directly evaluating the fluency and coordinative learning hypotheses described above. The learning and behavioral differences between the novel and standard tools may have been due to a lack of user-friendliness rather than having anything to do with questions of attention or coordination.

Focusing: While the interface designed to evaluate the fluency hypothesis was not effective, the experiment did provide evidence to support the focusing hypothesis. Students who used the unrestricted copy-paste tool performed worse when they recorded wordy notes, while the same was not true for other tools. It appears that selection-based note taking requires students to focus on the critical components of the ideas they are recording in order for it to be effective.

Coordination: There was no direct evidence to support the coordinative hypothesis in this study. However, all of the tools we have used so far have had students take notes in a separate window or sheet of paper. Therefore all get additional representations. A future study will address this limitation.

Descendents

Note-Taking: Coordination

Further Information

The following is a reference to a conference paper regarding our earlier study contrasting handwriting, typing, and copy-paste.

Bauer, A., Koedinger, K. Pasting and Encoding: Note-taking in Online Courses. IEEE International Conference on Advanced Learning Technologies (ICALT 2006), 5-7 July, Kerkrade, Netherlands.

- Bauer, A., Koedinger. K.R., Selection-Based Note-Taking Applications, ACM Symposium on Human Factors in Computing Systems 2007, in submission