Visual-verbal integration

Brief statement of principle

Visual-verbal integration principle: Instruction that includes both visual and verbal information leads to robust learning (the development of coherent, flexible knowledge representations) when the instruction supports learners as they coordinate information from both sources and the representations guide student attention to deep features.

Description of principle

Operational definition

Instruction encourages students to link or coordinate visual information (e.g., diagrams) and verbal information (e.g., text) by:

• supporting direct interaction with diagrams during problem solving (Butcher studyXXref)
• presenting diagrams that make explicit key features of an expert mental model (Davenport et al. studies)

Examples

In geometry, students need to connect the conceptual definition of a geometry principle (e.g., a verbal description of "Vertical Angles") with the relevant visual diagram features and configurations (e.g., the visual instantiation of "Vertical Angles" formed by two crossing lines where the angles share a common vertex but no common sides). Visual-verbal integration can be tested by having students analyze the appropriateness of geometry rules to a particular diagram.

Experimental support

Laboratory experiment support

Prior research has shown that students benefit from activities that coordinate both visual and verbal sources; these activities include verbal comparison of self-generated and ideal diagrams (Van Meter, 2001; Van Meter, Aleksic, Schwartz, & Garner, 2006) as well as dragging and dropping verbal information into a diagram to create an integrated representation (Bodemer, Ploetzner, Feuerlein, & Spada, 2004).

In vivo experiment support

Butcher and Aleven's (2007; submitted) in vivo research has demonstrated that the addition of interactive diagrams to an intelligent tutor in geometry supports deep understanding of geometry principles and long-term retention of problem-solving skills. The interactive diagrams were designed as a method to support visual-verbal integration; they allow students to work directly with the diagrams during problem solving. Results show that students who used the interactive diagrams are better able to work with new diagrams and geometry principles to 1) explain when and why geometry principles are inappropriately applied to a diagram, and 2) to explain how unsolvable problems could be made solvable. For more details on these studies, please see Contiguous Representations for Robust Learning (Aleven & Butcher) and Using Elaborated Explanations to Support Geometry Learning (Aleven & Butcher).

Butcher and Aleven also have been studying scaffolds that directly connect relevant visual and verbal information. Results of these studies are ongoing; for more information, please see Mapping Visual and Verbal Information: Integrated Hints in Geometry (Aleven & Butcher) and Visual Feature Focus in Geometry: Instructional Support for Visual Coordination During Learning (Butcher & Aleven).

Theoretical rationale

(These entries should link to one or more learning processes.)

References

Bodemer, D., Ploetzner, R., Feuerlein, I., & Spada, H. (2004). The active integration of information during learning with dynamic and interactive visualisations. Learning and Instruction, 14, 325-341.

Butcher, K., & Aleven, V. (2007). Integrating visual and verbal knowledge during classroom learning with computer tutors. In D.S. McNamara & J.G. Trafton (Eds.), Proceedings of the 29th Annual Cognitive Science Society (pp. 137-142). Austin, TX: Cognitive Science Society.

Butcher, K., & Aleven, V. (submitted). Diagram Interaction during Intelligent Tutoring in Geometry: Support for Knowledge Retention and Deep Transfer. Submitted to CogSci 2008.

Van Meter, P. (2001). Drawing construction as a strategy for learning from text. Journal of Educational Psychology, 93(1), 129-140.

Van Meter, P., Aleksic, M., Schwartz, A., & Garner, J. (2006). Learner-generated drawing as a strategy for learning from content area text. Contemporary Educational Psychology, 31, 142-166.

See also integration and coordination.