E-Learning Design Principles and Methods 2016

Course Details

Course number: 05-823

Semester: Fall 2015

Carnegie Mellon University

Class times

9:00 to 10:20 Tuesday & Thursday

Location

Gates Hillman Center (GHC) Room 5222

Instructor

Professor Ken Koedinger

Office: 3601 Newell-Simon Hall, Phone: 412-268-7667

Email: Koedinger@cmu.edu, Office hours by appointment

Course Prerequisites

To enroll you must either be in the Masters of Educational Technology and Applied Learning Science (METALS) or get the permission of the instructor.

Textbook and Readings

"E-Learning and the Science of Instruction: 3rd edition" by Ruth Colvin Clark and Richard E. Mayer.

Other readings will be assigned in class. See below.

Class URLs

Syllabus and useful links: learnlab.org/research/wiki/index.php/E-learning_Design_Principles_2014

For quizzes and reading reports go www.cmu.edu/blackboard.

Goals

This course is about e-learning design principles, the evidence and theory behind them, and how to apply these principles to develop effective educational technologies. It is organized around the book "e-Learning and the Science of Instruction: Proven Guidelines for Consumers and Designers of Multimedia Learning" by Clark & Mayer with further readings drawn from cognitive science, educational psychology, and human-computer interaction. You will learn design principles 1) for combining words, audio, and graphics in multimedia instruction, 2) for combining examples, explanations, practice and feedback in online support for learning by doing, and 3) for balancing learner versus system control and supporting student metacognition. You will read about the experiments that support these design principles, see examples of how to design such experiments, and practice applying the principles in your own educational technology design project.

Flipped Homework: Reading Quizzes and Reading Reports

You will have "flipped homework", a variation on the flipped classroom idea you might have heard of. Flipped homework is an assignment before a relevant class meeting rather than after it. It helps you to check your understanding of what you read, to practice to enhance your memory (we will talk about the "testing effect" in class), and to get a better sense of what you don't know so you are prepared to ask questions in class. It also helps instructors focus the class discussion to better avoid belaboring known points and pursue student needs and interests.

Before some class sessions, you will asked to do a quiz associated with the assigned book chapter. The quizzes will be on the Blackboard site (www.cmu.edu/blackboard, the course is listed as "Special Topics in HCI"). Before other class sessions, you will be asked to write "reading reports". We will use the discussion board on Blackboard. You should complete assigned quizzes or reading reports before 9am on the day of class.

For reading reports, the discussion forum post will usually direct you as to how to reply. If not otherwise directed, you should make two posts on the readings. Your two posts may be original or in response to another post (one of both is nice).

• Original posts should contain one or more of the following:
  • something you learned from the reading or slides
  • a question you have about the reading or slides or about the topic in general
  • a connection with something you learned or did previously in this or another course, or in other professional work or research
• Replies should be an on-topic, relevant response, clarification, or further comment on another student’s post.

In general, please come to class prepared to ask questions and give answers.

Laptop Policy

Given that class discussion is a major part of the course, laptops, cell phones, and smart phones are not to be used in class during Lecture days. Failure to listen to this will result in a reduction in your participation grade. During testing days (marked as such on the schedule), however, you will need your laptop.

Students have the option of using a laptop during presentations only if they are doing so to take notes and submit those notes to the full class for example on blackboard. To facilitate note taking during Lecture days, lecture slide handouts may be provided, if requested.

If interested in what educational research says about laptop use in class, or multi-tasking more generally, you might look at (available on the course BlackBoard):

• Fried, C. B. (2008). In-class laptop use and its effects on student learning. Computers & Education, 50, 906–914.

• Kirschner, P. A., & Merriënboer, J. J. V. (2013). Do learners really know best? Urban legends in education. Educational Psychologist, 48(3), 169–183. doi:10.1080/00461520.2013.80439

• Kraushaar, J. M., & Novak, D. C. (2010). Examining the affects [sic] of student multitasking with laptops during the lecture. Journal of Information Systems Education, 21(2), 241-251.

• Wood, E., Zivcakova, L., Gentile, P., Archer, K., De Pasquale, D., & Nosko, A. (2012). Examining the impact of off-task multi-tasking with technology on real-time classroom learning. Computers & Education, 58(1), 365-374. doi:10.1016/j.compedu.2011.08.02

Grading

• 35% Final Project Project assignment
  • Six parts of final project
  • Final project submission
• 5% E-Learning examples assignment
• 15% Midterm exam
• 15% Pre-class quizzes & reading reports
• 15% Final Exam
• 15% Class participation, including reading summary presentations

Class Schedule in Brief

• E-Learning Introduction 9-1- to 9-3
  • Sept 1 Overview; Examples Assignment; Project; 1.E-learning (The "1." indicates this is a chapter in the Clark & Mayer book)
  • Sept 3 2.How People Learn; KLI Framework Events; Project topic brainstorming
• Instructional Goals and Assessment 9-8 to 9-17
  • Sept 8 Determining instructional goals; Bloom's taxonomy; KLI KCs; Practice on goal setting
  • Sept 10 Standards & Assessment Frameworks; Evidence-based design; Practice assessment writing
  • Sept 15 Goal-setting Interviews: Structured Interviews, Contextual Inquiry; Practice interviewing
  • Sept 17 Online assessment; Practice e-assessment implementation
• Cognitive Task Analysis (CTA) 9-22 to 10-1
  • Sept 22 Rational & Empirical CTA via Think Alouds
  • Sept 24 Quantitative Cognitive Task Analysis: Difficulty Factors Assessment
  • Sept 29 CTA to improve model building & instructional design
  • Oct 1 Quantitative CTA via Data Mining
• Multimedia Principles 10-6 to 10-22
  • Oct 6 3.Evidence-based practice; KLI Learning & Instructional Events;
  • Oct 8 4.Multimedia Principle; 5.Contiguity Principle; Practice applying
  • Oct 13 6.Modality Principle & 7.Redundancy Principle; Practice applying
  • Oct 15 8.Coherence Principle & 9.Personalization Principle; Practice applying
  • Oct 20 Flex topic; Midterm review
  • Oct 22 Midterm exam Bring laptop to class
• Learning By Doing Principles 10-27 to 11-24
  • Oct 27 KLI & Selecting appropriate instructional principles
  • Oct 29 10.Segmenting and Pretraining; 11.Leveraging Examples in E-Learning
  • Nov 3 12.Does Practice Make Perfect; 14.Who’s in Control?
  • Nov 5 15.E-Learning to Build Problem Solving Skill; 16.Simulations and Games
  • Nov 10 17.Applying the Guidelines; KLI Review
  • Nov 12 [Guest topic: Options CSCL, Cognitive Mastery, Hint Factory, CTAT?] (need substitute for this class)
  • Nov 17 Peer review of instructional design
  • Nov 19 In vivo experimentation; A/B Testing
  • Nov 24 Flex topic; Presentation & Report Preparation
  • Nov 26 Thanksgiving, no class
• Final & Project Presentations 12-1 to 12-10
  • Dec 1 Final Exam Bring laptop to class
  • Dec 3 Project Presentations
  • Dec 8 Project Presentations
  • Dec 10 Project Presentations
• Final Project due Dec 18
• If needed: Final Exam Make-up - Thurs Dec 17, 1-4pm in GHC 5222

Class Schedule with Readings and Assignments

NOTE: This section is "living" -- parts will evolve as I get a better sense of your needs.

E-Learning Introduction 9-1 to 9-3

• 9-1 Course Objectives & Course Project; The boom in e-learning!
  • Reading (from course book): 1.e-Learning: Promise & Pitfalls (20 pages). This chapter is here (click to get) but order the book right now!
    • Pre-class quiz: Answer questions for Chpt1 Quiz on Blackboard
    • Slides for this chapter are here.
  • Class activity: Introduce your background and interests in e-learning
  • Assignment: Examples assignment is due next Mon, Sept 7. Please submit on blackboard.
  • Assignment: Project step 1 is due in 16 days on Thursday, 9-17
  • For next time:
    • BRING two screen shots of an e-learning example to next class
    • Review project step 1 and come with a preliminary project idea.
    • a) Do the readings & b) associated flipped homework (See next date for reading assignment)

• 9-3 How People Learn; KLI Framework events; Project topic brainstorming
  • Read Ch2. How Do People Learn from E-Courses (20 pages) You can get the chapter here this last time! Please order the book now if you have not!
  • Read sections 1-2 of KLI Framework paper (we will discuss other sections later)
    • Do the quiz for the readings (Quiz 2).
  • Class activity: Promises & pitfalls review of e-learning examples
    • BRING a print-out of an e-learning example to class
  • Class activity: Project idea discussion
    • Come prepared with a preliminary project idea
  • For next time:
    • a) Do the readings & b) associated flipped homework

Instructional Goals and Cognitive Task Analysis 9-8 to 9-17

• 9-8 Determining instructional goals; Bloom's taxonomy; KLI KCs; Practice on goal setting
  • Class activity: Review Project ideas and step 1 write-up requirements; consider assessment tasks
  • Class activity: goal setting
  • Reading: KLI section 3
  • Reading: Feldon paper
    • Do quizzes for the readings.

• 9-10 Standards & Assessents Frameworks; Evidence-based design; Practice assessment writing
  • Reading: Assessment design
  • Reading: Classic theory
  • Class activity: assessment writing

• 9-15 Goal-setting Interviews: Structured Interviews, Contexual Inquiry; Practice interviewing
  • Class activity: interviewing

• 9-17 Online assessment; Practice e-assessment implementation
  • Class activity: e-assessment implementation
  • DUE: P1: Context & Initial Resources
  • Assignment: P2 is due Oct 1

Cognitive Task Analysis (CTA) 9-22 to 10-1

• 9-22 Rational & Empirical CTA via Think Alouds
  • Reading: Lovett paper and Gomoll paper
  • Reading: Zhu & Simon paper
  • Optional Reading: Working Minds: A practitioner's Guide to Cognitive Task Analysis, Ch 1 and Ch 2
    • Do quizzes for the readings.

• 9-24 Quantitative Cognitive Task Analysis: Difficulty Factors Assessment
  • Reading: Heffernan paper
    • Do quizzes for the reading.
  • Come with an attempt at a model of one your task solutions and, ideally, with an initial draft of project step 2.

• 9-29 CTA to improve model building & instructional design
  • Reading: Clark_CTA_In_Healthcare_Chapter_2012
  • Optional readings:Working Minds Ch 5; Clark et al., 2007 Chapter proof
    • Do two posts for reading on Discussion Board

• 10-1 Quantitative CTA via Data Mining
  • Class activity: Work on P3. Analyzing your data
  • Reading: e-learning data to improvement (10 pages)
    • Do quizzes for e-learning data to improvement paper
  • DUE: P2: Identifying Goals & Online Assessment Creation
  • Assignment: P3 is due Oct 15

Multimedia Principles 10-6 to 10-22

• 10-6 Evidence-based practice; KLI Learning & Instructional Events
  • Class activity: Principles present in e-learning examples
  • Reading: 3.Evidence-based practice (18 pages)
  • Reading: KLI sect 4-5 (12 pages)
    • Do quizzes for the readings.

• 10-8 Multimedia Principle; Contiguity Principle; Practice applying
  • Reading: 4.Multi-media Principle (24 pages)
  • Reading: 5.Contiguity Principle (24 pages)
  • Optional Reading: Working Minds: A practitioner's Guide to Cognitive Task Analysis, Ch 1 and Ch 2
    • Do quizzes for the readings.

• 10-13 Modality Principle & Redundancy Principle; Practice applying
  • Class activity: Work on P3. How will you collect data?
  • Reading: 6.Modality Principle (18 pages)
  • Reading: 7.Redundancy Principle (18 pages)
    • Do quizzes for the readings.

• 10-15 Coherence Principle & Personalization Principle; Practice applying
  • Reading: 8.Coherence Principle (28 pages)
  • Reading: 9.Personalization Principle (26 pages)
    • Do quizzes for the readings.
  • Due: P3: Cognitive Task Analysis & Cognitive Model
  • Assignment: P4 is due Oct 29

• 10-20 Flex topic; Midterm Review

• 10-22 Midterm exam Bring laptop to class
  • Optional Readings:
    • Visual & Graphic Design for e-learning blog
    • Do Learners Really Know Best? Urban Legends in Education

Learning By Doing Principles 10-27 to 11-24

• 10-27 KLI & Selecting appropriate instructional principles
  • Reading: KLI sections 6-7
    • Do quizzes for the reading.
  • Assignment: P4 is due 10-29

• 10-29 Segmenting and Pretraining; Leveraging Examples in E-Learning
  • Reading: 10.Segmenting and Pretraining (18 pages)
  • Reading: 11.Leveraging Examples in E-Learning (28 pages)
    • Do quizzes for the readings.
  • DUE: P4: Initial Instructional Design
  • Assignment: P5 is due 11-19

• 11-3 Does Practice Make Perfect; Who’s in Control?
  • Reading: 12.Does Practice Make Perfect (28 pages)
  • Reading: 14.Who’s in Control? (30 pages)
    • Do quizzes for the readings.

• 11-5 E-Learning to Build Problem Solving Skill; Simulations and Games
  • Reading: 15.E-Learning to Build Problem Solving Skill (30 pages)
  • Reading: 16.Simulations and Games (32 pages)
    • Do quizzes for the readings.

• 11-10 Applying the Guidelines; KLI Review
  • Reading: 17.Applying the Guidelines (24 pages)
    • Do quizzes for the reading.

• 11-12 [Guest topic: Options CSCL, Cognitive Mastery, Hint Factory, CTAT?]

• 11-17 Peer review of instructional design

• 11-19 In vivo experimentation; A/B Testing
  • DUE: P5: Instructional Design Prototyping & Testing
  • Assignment: P6 is due 12-1

• 11-24Flex topic; Presentation & Report Preparation

• 11-26 Thanksgiving, no class

Final & Project Presentations 12-1 to 12-10

• 12-1 Final Exam Bring laptop to class
  • If needed: Final Exam Make-up - Thurs Dec 17, 1-4pm in GHC 5222
  • DUE: P6: Experimental Design
  • Assignment: Final Project is due 12-18. It should include the reflection statement (see the project assignment handout).

• 12-3 Project Presentations
• 12-8 Project Presentations
• 12-10 Project Presentations