Visual-verbal integration

Brief statement of principle

Visual-verbal integration principle: Instruction that includes both visual and verbal information leads to robust learning (the development of coherent, flexible knowledge representations) when the instruction supports learners as they coordinate information from both sources and the representations guide student attention to deep features.

Description of principle

Operational definition

Instruction encourages students to link or coordinate visual information (e.g., diagrams) and verbal information (e.g., text) by:

• Supporting direct interaction with diagrams during problem solving
  • For more information, see Butcher & Aleven studies: Contiguous Representations; Elaborated Explanations; Integrated Hints
• Presenting diagrams that make explicit key features of an expert mental model
  • For more information, see Davenport et al. studies)

Examples

In geometry, students need to connect the conceptual definition of a geometry principle (e.g., a verbal description of "Vertical Angles") with the relevant visual diagram features and configurations (e.g., the visual instantiation of "Vertical Angles" formed by two crossing lines where the angles share a common vertex but no common sides). Visual-verbal integration can be tested by having students analyze the appropriateness of geometry rules to a particular diagram.

Experimental support

Laboratory experiment support

Prior research has shown that students benefit from activities that coordinate both visual and verbal sources; these activities include verbal comparison of self-generated and ideal diagrams (Van Meter, 2001; Van Meter, Aleksic, Schwartz, & Garner, 2006) as well as dragging and dropping verbal information into a diagram to create an integrated representation (Bodemer, Ploetzner, Feuerlein, & Spada, 2004).

In vivo experiment support

Butcher and Aleven's (2007; submitted) in vivo research has demonstrated that the addition of interactive diagrams to an intelligent tutor in geometry supports deep understanding of geometry principles and long-term retention of problem-solving skills. The interactive diagrams were designed as a method to support visual-verbal integration; they allow students to work directly with the diagrams during problem solving. Results show that students who used the interactive diagrams are better able to work with new diagrams and geometry principles to 1) explain when and why geometry principles are inappropriately applied to a diagram, and 2) to explain how unsolvable problems could be made solvable. For more details on these studies, please see Contiguous Representations for Robust Learning (Aleven & Butcher) and Using Elaborated Explanations to Support Geometry Learning (Aleven & Butcher).

Butcher and Aleven also have been studying scaffolds that directly connect relevant visual and verbal information. Results of these studies are ongoing; for more information, please see Mapping Visual and Verbal Information: Integrated Hints in Geometry (Aleven & Butcher) and Visual Feature Focus in Geometry: Instructional Support for Visual Coordination During Learning (Butcher & Aleven).

Davenport et al. (2007, 2008) tested the role of visual-verbal integrate in chemistry instruction and found that instruction that includes diagrams and text only leads to learning gains when the representations are clearly aligned with an expert mental model. A knowledge decomposition of chemical equilibrium (informed by Lab Studies #1 and #2) as well as discussions with our Chemistry working group revealed that a key knowledge component of "progress of reaction" is left implicit in many types of traditional chemistry instruction. In one study two sets of online lectures were created by Prof. Yaron to determine if instruction that uses multiple representations to convey the notion of progress of reaction would lead to more robust learning of chemistry concepts. Traditional instruction described equilibrium using chemical notations and text, the New instruction described equilibrium using molecular diagrams depicting the progress of reaction. Transfer measures of open-ended responses and conceptual multiple choice questions were collected and revealed that diagrams that were aligned with the progress of reaction framework increased learning, particularly for low knowledge students.

Theoretical rationale

(These entries should link to one or more learning processes.)

Conditions of application

Instruction that promotes Visual-verbal integration will only be successful if students actively process information from both the pictures and text and if the informational content of pictures and text are clearly aligned with instructional objectives.

References

Bodemer, D., Ploetzner, R., Feuerlein, I., & Spada, H. (2004). The active integration of information during learning with dynamic and interactive visualisations. Learning and Instruction, 14, 325-341.

Butcher, K., & Aleven, V. (2007). Integrating visual and verbal knowledge during classroom learning with computer tutors. In D.S. McNamara & J.G. Trafton (Eds.), Proceedings of the 29th Annual Cognitive Science Society (pp. 137-142). Austin, TX: Cognitive Science Society.

Butcher, K., & Aleven, V. (submitted). Diagram Interaction during Intelligent Tutoring in Geometry: Support for Knowledge Retention and Deep Transfer. Submitted to CogSci 2008.

Davenport, J. L., Yaron, D., Klahr, D., & Koedinger, K. (2008). When do diagrams enhance learning? A framework for designing relevant representations. Paper accepted for the 2008 International Conference of the Learning Sciences, June 2008. download

Davenport, J.L., Klahr, D. & Koedinger (2007). The influence of diagrams on chemistry learning. Paper presented at the 12th Biennial Conference of the European Association for Research on Learning and Instruction. August 2007. download

Van Meter, P. (2001). Drawing construction as a strategy for learning from text. Journal of Educational Psychology, 93(1), 129-140.

Van Meter, P., Aleksic, M., Schwartz, A., & Garner, J. (2006). Learner-generated drawing as a strategy for learning from content area text. Contemporary Educational Psychology, 31, 142-166.

See also integration and coordination.