Difference between revisions of "E-learning Design Principles 2013"

From LearnLab
Jump to: navigation, search
(New page: ==DRAFT DRAFT E-Learning Design Principles 05-899 DRAFT DRAFT== Fall 2013 Syllabus Carnegie Mellon University ====Class times==== 1:30 to 2:50 Tuesday & Thursday ====Location==== 5222...)
 
(→‎Course Details)
 
(37 intermediate revisions by the same user not shown)
Line 1: Line 1:
−
==DRAFT DRAFT  E-Learning Design Principles 05-899 DRAFT DRAFT==
+
====Course Details====
−
Fall 2013 Syllabus Carnegie Mellon University
+
More recent version at [[E-Learning Design Principles 2014]]
 +
 
 +
Course number & official name: 05-899 Special Topics in HCI: E-Learning Design Principles
 +
 
 +
Semester: Fall 2013  
 +
 
 +
Carnegie Mellon University
 
   
 
   
−
====Class times====
+
=====Class times=====
 
1:30 to 2:50 Tuesday & Thursday
 
1:30 to 2:50 Tuesday & Thursday
  
−
====Location====
+
=====Location=====
 
5222 Gates-Hillman Center (GHC)
 
5222 Gates-Hillman Center (GHC)
  
−
====Instructor====
+
=====Instructor=====
 
Professor Ken Koedinger
 
Professor Ken Koedinger
  
Line 15: Line 21:
 
Email: Koedinger@cmu.edu, Office hours by appointment
 
Email: Koedinger@cmu.edu, Office hours by appointment
  
−
====Class URLs====   
+
=====Course Prerequisites=====
 +
To enroll you must either be in the Masters of Educational Technology and Applied Learning Science (METALS) or get the permission of the instruction.
 +
 
 +
=====Textbook and Readings=====
 +
"E-Learning and the Science of Instruction: 3rd edition" by Ruth Colvin Clark and Richard E. Mayer.
 +
 +
Other readings will be assigned in class.  See below.
 +
 
 +
=====Class URLs=====   
 
Syllabus and useful links: [http://learnlab.org/research/wiki/index.php/E-learning_Design_Principles_2013 learnlab.org/research/wiki/index.php/E-learning_Design_Principles_2013]
 
Syllabus and useful links: [http://learnlab.org/research/wiki/index.php/E-learning_Design_Principles_2013 learnlab.org/research/wiki/index.php/E-learning_Design_Principles_2013]
  
−
For reading reports: [http://www.cmu.edu/blackboard/ www.cmu.edu/blackboard]
+
For quizzes and reading reports go [http://www.cmu.edu/blackboard/ www.cmu.edu/blackboard]. The course is listed as "Special Topics in HCI".
  
 
====Goals====
 
====Goals====
 +
This course is about e-learning design principles, the evidence and theory behind them, and how to apply these principles to develop effective educational technologies. It is organized around the book "e-Learning and the Science of Instruction: Proven Guidelines for Consumers and Designers of Multimedia Learning" by Clark & Mayer with further readings drawn from cognitive science, educational psychology, and human-computer interaction.  You will learn design principles 1) for combining words, audio, and graphics in multimedia instruction, 2) for combining examples, explanations, practice and feedback in online support for learning by doing, and 3) for balancing learner versus system control and supporting student metacognition. You will read about the experiments that support these design principles, see examples of how to design such experiments, and practice applying the principles in educational technology development.
  
 +
====Flipped Homework: Reading Reports and Reading Quizzes====
  
−
====Course Prerequisites====
+
You will have "flipped homework", a variation on the flipped classroom idea you might have heard of. Flipped homework is an assignment before a relevant class meeting rather than after it. It helps you to check your understanding of what you read, to practice to enhance your memory (we will talk about the "testing effect" in class), and to get a better sense of what you don't know so you are prepared to ask questions in class. It also helps instructors focus the class discussion to better avoid belaboring known points and pursue student needs and interests.
−
To enroll you must be in the Masters of Educational Technology and Applied Learning Science (METALS) or get the permission of the instruction.
 
  
−
====Textbook and Readings====
+
Before some class sessions, you will asked to do a quiz associated with the assigned book chapter.  The quizzes will be on the Blackboard site ([http://www.cmu.edu/blackboard/ www.cmu.edu/blackboard], the course is listed as "Special Topics in HCI"). Before other class sessions, you will be asked to write "reading reports".  We will use the discussion board on Blackboard. You should complete assigned quizzes or reading reports ''before 9am''' on the day of class.
−
"E-Learning and the Science of Instruction: 3rd edition" by Ruth Colvin Clark and Richard E. Mayer.  You can find it at []
 
  
−
Other readings will be assigned in class.  See below.
+
For reading reports, the discussion forum post will usually direct you as to how to reply.   
−
 
+
If not otherwise directed, you should make '''two posts''' on the readings. Your ''two'' posts may be original or in response to another post (one of both is nice).
−
====Flipped Homework: Reading Reports and Pre-Class Assignments====
 
−
 
 
−
We are often going to implement "flipped homework", a variation on the flipped classroom idea you might have heard of.  Flipped homework is an assignment before a relevant class meeting rather than after it.  It helps students (you!) to "problematize" the topic -- to get a better
 
−
sense of what you don't know and what questions you have. It helps instructors focus the class discussion to better avoid belaboring what students already know and to better pursue student needs and interests.
 
−
 
 
−
Students will be asked to write "reading reports" before most class sessions.  We will use the discussion board on Blackboard ([http://www.cmu.edu/blackboard/ www.cmu.edu/blackboard]) for this purpose.   
 
−
 
 
−
Unless otherwise directed by instructors, students should make '''two posts''' on the readings '''before 9am''' on the day of class that those readings are due.  If slides for the class are available, please review these as well.
 
−
 
 
−
These posts serve multiple purposes: 1) to improve your understanding and learning from the readings, 2) to provide instructors with insight into what aspects of the readings merit further discussion, either because of student need or interest, and 3) as an incentive to do the readings before class!
 
−
 
 
−
In general, please come to class prepared to ask questions and give answers.
 
−
 
−
Your ''two'' posts may be original or in response to another post (one of both is nice).
 
 
*Original posts should contain one or more of the following:
 
*Original posts should contain one or more of the following:
 
**something you learned from the reading or slides
 
**something you learned from the reading or slides
 
**a question you have about the reading or slides or about the topic in general
 
**a question you have about the reading or slides or about the topic in general
 
**a connection with something you learned or did previously in this or another course, or in other professional work or research
 
**a connection with something you learned or did previously in this or another course, or in other professional work or research
−
 
 
*Replies should be an on-topic, relevant response, clarification, or further comment on another student’s post.
 
*Replies should be an on-topic, relevant response, clarification, or further comment on another student’s post.
  
−
You may be asked to do other activities before class, such as answer questions on-line using the [http://assistment.org Assistment system], parts of the an [http://oli.web.cmu.edu/openlearning/ OLI course], or beginning work on an assignment.  That way you can come to class with a better appreciation for what you do not understand and need to learn.
+
In general, please come to class prepared to ask questions and give answers.
  
 
====Grading====
 
====Grading====
  
−
There will be assignments associated with each section of the course.  Grades will be determined by your performance on these assignments, by before-class preparation activities including reading reports, by your participation in class, and by a final paper.
+
* 55% Final Project
−
 
+
** 45% Six parts of final project, 7.5% each
−
* Course work
+
** 10% Final project submission
−
** 30% Before-class preparation, including reading reports, and in-class participation 
+
* 5% E-Learning examples assignment
−
** 40% Assignments
+
* 10% Peer review and feedback
−
* Project & final paper - Due May 10.
+
* 10% Pre-class quizzes & reading reports
−
** 30% Design a new study based on one or more of these methods that pushes your own research in a new direction.
+
* 10% Chapter summary 
−
:# Apply a method from the class to your research. You should not choose a method that you already know well.
+
* 10% Class participation
−
:# Think of it as writing a grant proposal. Because some methods will be introduced after the project proposal date, we are open to a modification in your project to apply the newly introduced method.  But, please check with us to get feedback and approval on a proposed change.
 
−
:# No more than 15 double-spaced pages. Be efficient. Space is always limited in academic publications and you will find it useful to learn to include only what is important. Since this is styled as a grant proposal, please include some literature review and discussion of significance of the area you want to investigate. You should also briefly detail plans for participants, explain specifically how you will apply the method, and describe how you will analyze the data.
 
  
 
====Class Schedule in Brief====  
 
====Class Schedule in Brief====  
−
* Course Intro: Formulating Good Research Questions: Jan 15 (T)
+
*E-Learning Introduction 8-27 to 9-5
−
* Cognitive Task Analysis 1: Jan 17, 22, 24 (RTR)
+
**Aug 27 Overview; Examples Assignment; Project
−
* Video and Verbal Protocol Analysis: Jan 29, 31, Feb 5,7,12,14 (TRTRTR)
+
**Aug 29 1.E-learning; KLI Framework events (The "1." indicates this is a chapter in the Clark & Mayer book)
−
** Guest Instructors: Marsha Lovett & Carolyn Rose
+
**Sept 3 2.How People Learn; KLI KC's
−
* Cognitive Task Analysis 2: Feb 19, 21 (TR)
+
**Sept 5 3.Evidence-based practice; KLI Learning & Instructional Events
−
* Educational Measurement & Psychometrics: Feb 26, 28, Mar 5 (TRT)
+
*Instructional Goals and Cognitive Task Analysis 9-10 to 9-17  
−
** Guest Instructor: Brian Junker
+
**Sept 10 Determining instructional goals (tasks)
−
* Educational Design Research: Mar 7 (R)
+
**Sept 12 Guest lecture
−
* NO CLASS – Spring break, Mar 12, 14 (TR)
+
**Sept 17 Discovering learning objectives (KCs) & Rational Cognitive Task Analysis
−
* Surveys, Questionnaires, Interviews: Mar 19, 21 (TR)
+
*Multimedia Principles and Cognitive Task Analysis 9-19 to 10-17
−
** Guest Instructor: Sara Kiesler
+
**Sept 19 4.Multi-media Principle
−
* Educational Data Mining & Learning Curves: March 26, 28, Apr 2 (TRT)
+
**Sept 24 Empirical Cognitive Task Analysis: Think aloud
−
* Flex day: Apr 4 (R)
+
**Sept 26 5.Contiguity Principle
−
* Educational Data Mining & Causal Inference: Apr 9, 11, 16 (TRT)
+
**Oct 1 CTA: DFA & Model building
−
** Guest Instructor: Richard Scheines
+
**Oct 3 6.Modality Principle & 7.Redundancy Principle
−
* NO CLASS – Spring Carnival, Apr 18 (R)
+
**Oct 8 CTA & Designing Assessments for Continual Improvement
−
* Experimental Methods: Apr 23, 25, 30 (TRT)
+
**Oct 10 Midterm review; Flex topic
−
* Wrap-up: May 2 (R)
+
**Oct 15 8.Coherence Principle
 +
**Oct 17 9.Personalization Principle
 +
*Learning By Doing Principles 10-22 to 11-19  
 +
**Oct 22 10.Segmenting and Pretraining
 +
**Oct 24 KLI & Selecting appropriate instructional principles
 +
**Oct 29 11.Leveraging Examples in E-Learning
 +
**Oct 31 12.Does Practice Make Perfect
 +
**Nov 5 13.Learning Together Virtually
 +
**Nov 7 14.Who’s in Control?
 +
**Nov 12 15.E-Learning to Build Problem Solving Skill
 +
**Nov 14 16.Simulations and Games
 +
**Nov 19 17.Applying the Guidelines
 +
*Project Presentations 11-21 to 12-5
 +
**Nov 21 Project Presentations
 +
**Nov 26 Project Presentations
 +
**Nov 28 Thanksgiving, no class
 +
**Dec 3 Project Presentations
 +
**Dec 5 Project Presentations
 +
*Final Project due Dec 13
  
 
====Class Schedule with Readings and Assignments====  
 
====Class Schedule with Readings and Assignments====  
  
−
'''NOTE:''' This is a "living" document.  It carries over some elements from the past course offering that may get changed before the scheduled class period.
+
'''NOTE:''' This section is "living" -- it will grow and change as the semester goes on.
−
 
 
−
=====Course Intro & Formulating Good Research Questions (Koedinger)=====
 
−
*1-15
 
−
**See your email or [http://www.cmu.edu/blackboard/ www.cmu.edu/blackboard] for the pre-class assignment.
 
−
**[[Media:CourseIntroGoodQuestions13.pdf|Lecture slides]]
 
−
**Read Trochim Chapter 1, particularly sections 1-2d and 1-4. See above for how to get the book -- [[Media:Trochim-Ch01.pdf|but here's Chapter1]]
 
−
**[Optional (re)reading] Nathan, M., & Alibali, M. (2010). Learning sciences.  WIREs Cognitive Science.  [[Media:Nathan&Alibali_2010_WIREs_LS.pdf|PDF]]
 
−
 
 
−
=====Cognitive Task Analysis (Koedinger) =====
 
−
*1-17
 
−
**Zhu, X. & Simon, H. A. (1987). Learning mathematics from examples and by doing. Cognition and Instruction, 4(3), 137-166.  [[Media:Zhu&Simon-1987.pdf|Zhu&Simon-1987.pdf]]
 
−
**Do a couple short assignments here:  http://Assistment.org.  Please create and an account, click on "Tutor", "Enroll in a class", select "Ken Koedinger" and "Educational Research Methods".
 
−
**Slides: [[Media:CTA1-2013.pdf|CTA1-2013.pdf]]
 
−
**[Optional reading] Zhu X., Lee Y., Simon H.A., & Zhu, D. (1996). Cue recognition and cue elaboration in learning from examples. In Proceedings of the National Academy of Sciences 93, (pp. 1346±1351).  [[Media:PNAS-1996-Zhu-Simon.pdf|PNAS-1996-Zhu-Simon.pdf]]
 
−
*1-22
 
−
**Clark, R. E., Feldon, D., van Merriënboer, J., Yates, K., & Early, S. (2007). Cognitive task analysis: In J. M. Spector, M. D. Merrill, J. J. G. van Merriënboer, & M. P. Driscoll (Eds.), Handbook of research on educational communications and technology (3rd ed., pp. 577–593). Mahwah, NJ: Lawrence Erlbaum Associates. [[Media:Clarketal2007-CTAchapter.pdf|Clarketal2007-CTAchapter.pdf]]
 
−
***One point of reflection for you on the Clark et al reading is to compare and contrast the Cognitive Task Analysis (CTA) methods and output representations recommended with the approach taken by Zhu & Simon.  Also, note their examples and claims about the power of CTA for improving instruction.  (If you saw Bror Saxberg's PIER talk last year, you may have heard that Kaplan is using CTA, with Clark's advice, to revise and improve their courses.) 
 
−
**Chapter 2: How Experts Differ From Novices in Bransford, J. D., Brown, A., & Cocking, R. (2000). (Eds.), How people learn: Mind, brain, experience and school (expanded edition). Washington, DC: National Academy Press. [[Media:HowPeopleLearnCh2.pdf|HowPeopleLearnCh2.pdf]]
 
−
***Besides being an interesting read, a key point of this reading is the nature of expert knowledge (declarative and procedural) and how it is highly "conditionalized".  How is this claim similar or different from Zhu & Simon?  The notion of adaptive expertise is also important and interesting.
 
−
***As you read the 1-22 and 1-24 readings, be thinking about steps you could take to do a cognitive task analysis, empirical and rational, in a domain of your interest. Think about what tasks you would use, what CTA technique(s), and how might represent the output of your analysis.
 
−
**Slides: [[Media:CTA2-2013.pdf|CTA2-2013.pdf]]
 
−
*1-24
 
−
**Aleven, V., McLaren, B., Roll, I., & Koedinger, K. R. (2004). Toward tutoring help seeking: Applying cognitive modeling to meta-cognitive skills. In J.C. Lester, R.M. Vicari, & F. Parguacu (Eds.) Proceedings of the 7th International Conference on Intelligent Tutoring Systems, 227-239. Berlin: Springer-Verlag. [[Media:AlevenITS2004.pdf|AlevenITS2004.pdf]]
 
−
**Klahr, D., & Carver, S.M. (1988). Cognitive objectives in a LOGO debugging curriculum: Instruction, learning, and transfer. Cognitive Psychology, 20, 362-404. [[Media:Klahr&carver88.pdf|Klahr&carver88.pdf]]
 
−
**Siegler, R.S. (1976).  Three aspects of cognitive development. Cognitive Psychology, 8 (4), 481-520, Elsevier. [[Media:Siegler76.pdf|Siegler76.pdf]]
 
−
***Pick '''one''' of these readings to focus on and skim the other two.  Target your first post on that reading (and make clear which one it was).  Your second post can be on any of the three. These readings illustrate the use of Cognitive Task Analysis (CTA) outside of math domains. The Aleven et al reading provides an example of a CTA at the level of metacognitive skills.  The Siegler reading shows a CTA dealing with younger kids.  The Klahr & Carver reading shows how CTA can facilitate the design of instruction that achieves a substantial level of transfer.  When you skim all three, pay particular attention to 1) what are tasks the authors are analyzing, 2) what is their goal, 3) what is(are) the method(s) of analysis, and 4) how do the authors represent the output of their analysis: Do they use any of production rules, goal trees, semantic nets, hierarchical task models, or other?
 
−
**In the first forum (where you posted one of your research topics), reply to your thread with a post that describes an example task that you could productively analyze in your domain of interest. You might also indicate some variations on the task that might help reveal what is most challenging for learners.
 
−
**Slides: [[Media:CTA3-2013.pdf|CTA3-2013.pdf]]
 
−
*Other possible readings:
 
−
**Newell & Simon [[Media:Human_Problem_Solving.pdf|Human_Problem_Solving.pdf]]
 
−
**Lovett [[Media:Lovett01CandI.pdf|Lovett01CandI.pdf]]
 
−
 
 
−
=====Video and Verbal Protocol Analysis (Lovett, Rosé) =====
 
−
 
 
−
The plan for these six sessions in 2013, 1-29 to 2-14, is in [[Media:PIERResearchMethodsPlan2013.doc|this document]].
 
−
 
 
−
By the end of this module, students should be able to:
 
−
*Explain what is involved in collecting and analyzing verbal data (including both “hand” and automatic approaches to analysis)
 
−
*Recognize when – and explain why – protocol analysis is/is not appropriate to particular research situations.
 
−
*Apply protocol analysis methods to already collected and segmented data.
 
−
 
 
−
Besides reading and discussing articles, students will complete a coding scheme design assignment.
 
−
 
 
−
Four parts of this assignment will be done as homework or in-class work:
 
−
*Part A (homework): Between sessions 2 and 3, propose one or more hypotheses and think about how you could use protocol analysis on the given data set to evaluate those hypotheses.
 
−
*Part B (homework): By session 5, develop a short coding manual and apply your coding scheme to a subset of the provided data.  Bring 2 printouts to class.  Also install LightSIDE software on your laptop and make sure it runs (http://www.cs.cmu.edu/~emayfiel/side.html).
 
−
*In class Part C: In session 5, swap coding manuals with a classmate and use their coding manual to code the same data they have coded (but not looking at their codes!), and measure reliability.
 
−
*Part D (homework): For session 6, prepare data for automatic coding, and bring soft-copy to class along with your laptop.
 
−
 
 
−
 
 
−
*Session 1[Jan 29]: Overview of Protocol Analysis
 
−
 
 
−
**In this introductory discussion, we will explore the basics of collecting verbal protocol data as well as a high-level view of what’s involved in analyzing such data. We will explore different uses of verbal data.
 
−
 
 
−
**Chi, M. T. H. (1997).  Quantifying qualitative analyses of verbal data: A practical guide.  The Journal of the Learning Sciences, 63), 271-315.
 
−
[[http://chilab.asu.edu/papers/Verbaldata.pdf]]
 
−
 
 
−
**Discussion Questions:
 
−
***What are the main contrasts between the approach Chi advocates for analysis of verbal data and how she presents verbal protocol analysis?
 
−
***What can be gained from using these approaches?  Which if either do you have experience with, and if so, can you explain that experience?
 
−
***How does Chi present these methodologies as complementary to more formally quantitative methodologies?
 
−
 
 
−
 
 
−
*Session 2[Jan 31 Carolyn]: Protocol Analysis of Collaborative Learning Discussions
 
−
 
 
−
**In this session we will explore the connection between talk and learning, specifically investigating how stylistic aspects of language use enable or constrain articulation of ideas at different levels of abstraction, and how they affect how students position themselves or are positioned within an academic discourse. 
 
−
 
 
−
**Howley, I., Adamson, D., Dyke, G., Mayfield, E., Beuth, J. & Rosé, C. (2012).  Group Composition and Intelligent Dialogue Tutors for Impacting Students’ Academic Self-efficacy. Proceedings of the Intelligent Tutoring Systems Conference [[http://www.cs.cmu.edu/~emayfiel/application_papers/120113ITS12_ikh_07cpr.pdf]].
 
−
 
 
−
**Howley, I., Mayfield, E. & Rosé, C. P. (2013).  Linguistic Analysis Methods for Studying Small Groups, in Cindy Hmelo-Silver, Angela O’Donnell, Carol Chan, & Clark Chin (Eds.) International Handbook of Collaborative Learning, Taylor and Francis, Inc.[[http://www.learnlab.org/research/wiki/images/5/58/Chapter-Methods-Revised-Final.pdf]]
 
−
 
 
−
**Coding Manual for Negotiation [[http://www.learnlab.org/research/wiki/images/9/9c/Negotiation_10.pdf]]
 
−
 
 
−
*Discussion Questions:
 
−
**What do you see as the advantages and disadvantages of adopting methods from linguistics for the analysis of verbal data from studies of student learning?
 
−
**In the chapter, the role of discussion in learning as it is conceptualized within a variety of theoretical frameworks was compared and contrasted.  Which do you agree most with and why?
 
−
**Pick one of the conversation extracts from the chapter and critique the provided analysis from the perspective of your chosen theoretical framework.
 
−
**How could protocol analysis be used to shed light on what was happening in the Howley et al., 2012 study?
 
−
 
 
−
*Session 3[Feb 5 Marsha]: Practical aspects of analyzing verbal data
 
−
 
 
−
**In this session we will break down the process of designing a coding scheme into practical steps.
 
−
 
 
−
**Gihooly, K. J., Fioratou, E., Anthony, S. H., Wynn, V. (2007).  Divergent thinking: Strategies and executive involvement in generating novel uses for familiar objects, British Journal of Psychology, 98, pp 611-625. [[http://www.learnlab.org/research/wiki/images/c/c9/GihoolyEtAl2007.pdf]]
 
−
 
 
−
**van Someren, M. W., Barnard, Y. F., & Sandberg, J. A. C. (1994).The Think Aloud Method: A Practical Guide to Modelling Cognitive Processes. New York: Academic Press.  Chapter 7 [[http://www.learnlab.org/research/wiki/images/archive/6/63/20130125191704%21VanSch7.pdf]]
 
−
 
 
−
*Discussion Questions:
 
−
**What, if any, of the steps involved in protocol analysis did you find confusing?
 
−
**Which of these steps would you say are most methodologically challenging? most theoretically important?
 
−
**How might the steps differ for individual, talk-aloud data vs. collaborative, chat data?
 
−
 
 
−
*Session 4[Feb 7 Carolyn]: Methodological considerations related to manual and automatic analysis
 
−
 
 
−
**Here we will discuss issues related to reliability and validity, and efficiency of analysis.  We will also contrast different types of protocol analyses, namely categorical types of analyses versus word counting approaches.
 
−
 
 
−
**Rosé, C. P., Wang, Y.C., Cui, Y., Arguello, J., Stegmann, K., Weinberger, A., Fischer, F., (2008).  Analyzing Collaborative Learning Processes Automatically: Exploiting the Advances of Computational Linguistics in Computer-Supported Collaborative Learning, International Journal of Computer Supported Collaborative Learning [[http://www.learnlab.org/research/wiki/images/0/0e/Rose_Analyzing_Collaborative.pdf]]
 
−
 
 
−
*Discussion Questions:
 
−
**What was the most surprising result you read about in the paper?  How do the capabilities you read about compare with what you would expect to be able to do with automatic analysis technology?
 
−
**What role can you imagine automatic analysis of verbal data playing in your research?  Where would it fit within your research process?
 
−
**What do you think is the most important caveat related to automatic analysis described in the paper?
 
−
 
 
−
*Session 5[Feb 12 Marsha]: Inter-Rater Reliability and When to Use Protocol Data
 
−
 
 
−
**In this lecture, we will discuss issues of reliability for protocol data (how to compute Cohen’s kappa and how to resolve coding disagreements). We will also discuss the conditions under which verbal protocol data are/are not appropriate.
 
−
 
 
−
**Ericsson, K. A., & Simon, H. A. (1993). Protocol Analysis (pp. 1-31). Cambridge, MA: The MIT Press. [Introduction and Summary][[http://www.learnlab.org/research/wiki/images/archive/b/b8/20130125181231%21ProtAna1.pdf]]
 
−
**Ericsson, K. A., & Simon, H. A. (1993). Protocol Analysis (pp. 78-107). Cambridge, MA: The MIT Press. [Effects of Verbalization] [[http://www.learnlab.org/research/wiki/images/archive/f/fe/20130125181401%21ProtAnalysis2.pdf]]
 
−
 
 
−
 
 
−
*Discussion Questions:
 
−
**What are the key features that make verbal protocols appropriate/not?
 
−
**What can researchers do to collect and analyze such data most effectively?
 
−
 
 
−
*Session 6[Feb 14 Carolyn and Marsha]: Tools For Supporting Protocol Analysis
 
−
 
 
−
**In this session we will introduce some new technology for facilitating protocol analysis tasks.  Students will gain hands on experience with a new technology called SIDE Tools [[http://www.cs.cmu.edu/~emayfiel/side.html]].  You will work with the data you coded in the last session.  Please read the user’s manual.
 
−
 
 
−
*Discussion Questions:
 
−
**What evidence do you as a human use to distinguish between the codes in your coding scheme?  How much of this evidence do you think a computer would be able to take advantage of?
 
−
**Looking at your coded data, which aspects do you predict will be easy to automatically code, and which do you think will be too hard?
 
−
 
 
−
=====Cognitive Task Analysis - Revisited (Koedinger) =====
 
−
*2-19 
 
−
**Do one post on [[Media:Applying-CTA-assignment.docx|this assignment]] and a second post on the reading.
 
−
**In addition to think aloud, another empirical approach to Cognitive Task Analysis is to compare student performance on a space of similar tasks designed to test specific hypotheses about the knowledge demands of those tasks.  We have called this approach "Difficulty Factors Assessment" and the Koedinger & Nathan paper is an early example. While the assignment is a rational CTA, note the similarity in the logic of contrast used in Difficulty Factors Assessment and the contrast between the two tasks or solutions in the assignment. Skim Koedinger & MacLaren to see another example of a production rule model and of a method of quantitative evaluation of that model by fitting it to coding categories from a solution protocol analysis. 
 
−
**Koedinger, K.R. & Nathan, M.J. (2004).  The real story behind story problems: Effects of representations on quantitative reasoning.  ''The Journal of the Learning Sciences, 13'' (2), 129-164. [[Media:Koedinger-Nathan-LS04.pdf|Koedinger-Nathan-LS04.pdf]]
 
−
**Optional:  Koedinger, K.R., & MacLaren, B. A. (2002).  Developing a pedagogical domain theory of early algebra problem solving.  CMU-HCII Tech Report 02-100.  Accessible via http://reports-archive.adm.cs.cmu.edu/hcii.html [[Media:KoedingerMacLaren02.pdf|KoedingerMacLaren02.pdf]]
 
−
 
 
−
*2-21
 
−
**Koedinger, K.R. & McLaughlin, E.A. (2010). Seeing language learning inside the math: Cognitive analysis yields transfer. In S. Ohlsson & R. Catrambone (Eds.), ''Proceedings of the 32nd Annual Conference of the Cognitive Science Society.'' (pp. 471-476.) Austin, TX: Cognitive Science Society. [[Media:Koedinger-mclaughlin-cs2010.pdf|Koedinger-mclaughlin-cs2010.pdf]]
 
−
*Other optional readings
 
−
**Rittle-Johnson, B. & Koedinger, K. R. (2001). Using cognitive models to guide instructional design: The case of fraction division: In Proceedings of the Twenty-Third Annual Conference of the Cognitive Science Society, (pp. 857-862). Mahwah,NJ: Erlbaum. [[Media:Rittle-Johnson-Koedinger-cogsci01.pdf|Rittle-Johnson-Koedinger-cogsci01.pdf]]
 
−
**Koedinger, K. R., Corbett, A. C., & Perfetti, C. (2012).  The Knowledge-Learning-Instruction (KLI) framework: Bridging the science-practice chasm to enhance robust student learning. ''Cognitive Science''. [[Media:KLI-paper-v5.13.pdf|KLI-paper-v5.13.pdf]]
 
  
−
=====Psychometrics, reliability, Item Response Theory (Junker)=====
+
=====''E-Learning Introduction 8-27 to 9-5''=====  
  
−
*NEW ASSIGNMENTS [Plans for these classes were communicated by Brian Junker via email.]
+
*8-27 Overview, course project, your interests
 +
**Class activity: Discuss your interests in e-learning
 +
**Assignment: [[Media:edtech-example-review-assignment.docx|Examples (click to get)]] is due next Thursday, 9-5
 +
***BRING two screen shots of your first example to ''next'' class
 +
**Assignment: [[Media:E-learning-project-assignment-2013.docx|Project]] step 1 is due in 16 days on Thursday, 9-12
 +
**NOTE: See reading assignment for next time on next date.
  
−
*2-26
+
*8-29 E-learning intro and KLI Framework events [[Media:L01-e-learning-promises-pitfalls.pptx|(Click here for slides)]]
 +
**Class activity: Promises & pitfalls review of e-learning examples
 +
***BRING two screen shots of your first example to this class
 +
**Reading (from course book): 1.e-Learning: Promise & Pitfalls (28 pages). [[Media:E-Learning-Ch1.pdf|You can get this chapter here this time]] but order the book right now!
 +
***Pre-class quiz: Answer questions for Chpt1 Quiz on Blackboard
 +
**For next time:
 +
***BRING two screen shots of your ''second'' example to this class
 +
***Review project step1 and come with a preliminary project idea.  You might write some thoughts down, but you do not need to hand anything in.
 +
***a) Do the two readings, b) associated quiz & c) discussion board post on Blackboard
  
−
**Quick introduction to the R statistical language
+
*9-3 How People Learn and KLI Knowledge Components [[Media:L02-ELDP-how-people-learn.pptx|(Slides)]]
 +
**Read Ch2.How Do People Learn from E-Courses (20 pages) [[Media:E-Learning-Ch2.pdf|You can get this chapter here this '''last''' time!]]
 +
***Pre-class quiz: Answer questions for Chpt2 Quiz on Blackboard (5 minutes)
 +
**Read [[Media:KLI-Framework-KoedingerCorbettPerfetti2012.pdf|KLI Framework paper]] sections 1-3 (18 pages)
 +
***Make one post to Blackboard -- see questions in Forum introduction
 +
**Class activity: KC type in e-learning examples
 +
***BRING two screen shots of your ''second'' example to this class.
 +
**Class activity: Project idea discussion
 +
***Come prepared with a preliminary project idea
  
−
**Please complete and bring comments & questions to class on Tues Feb 28.
+
*9-5 Evidence-based practice and KLI Learning & Instructional Events [[Media: (Slides)]]
−
**Please download research_methods_r_assignment.zip from http://www.stat.cmu.edu/~brian/PIER-methods/.  The Zip file contains three further files:
+
**Reading: 3.Evidence-based practice (18 pages)
−
*** R-preassignment.pdf - instructions for this assignment
+
***Pre-class quiz: Answer questions for Chpt3 Quiz on Blackboard
−
*** r-tutorial-1.R - examples of statistical things that you will do in R, for this assignment
+
**Reading: KLI sect 4-5 (12 pages)
−
*** thermo11_data_integrated.csv - a data set for the examples.
+
***Make one post to Blackboard -- see questions in Forum introduction
 +
**Class activity: Principles present in e-learning examples
 +
**DUE: [[Media:edtech-example-review-assignment.docx|Examples assignment]] is due at beginning of class. Please submit on blackboard.
  
−
*2-28
+
=====Instructional Goals and Cognitive Task Analysis 9-10 to 9-17=====
  
−
1. From Trochim:  
+
*9-10 Goals, assessment tasks, cognitive task analysis, and instructional design
 +
**Class activity: Review Project ideas and step 1 write-up requirements; consider assessment tasks
 +
**Reading: [[Media:Feldon_Timmerman_etal_2010.pdf|Feldon paper]]
 +
***Posts: Do two posts on the Feldon reading.
  
−
  A. Chapter 3 - the vocabulary of measurement
+
*9-12 Cognitive Task Analysis and Think Alouds by guest lecturer Vincent Aleven
−
         
+
**DUE: Project step P1: Domain, Context & Initial Resources
−
  B. Chapter 5 - on constructing scales (it's ok to focus
+
**Assignment: Project step P2 is due on 9-26
−
      on the material up through sect 5.2a; the rest is
+
**Reading: [[Media:Lovett98.pdf|Lovett paper]] and [[Media:Gomoll-90.pdf|Gomoll paper]]
−
      more of a skim [but I'd be happy to talk about that
+
***Posts: Do two posts (total) on the Lovett and Gomoll papers.
−
      in class also])
 
  
−
2. On item response theory (IRT), a set of statistical models that are used
+
*9-17 Discovering learning objectives (KCs) and Rational Cognitive Task Analysis
−
to construct scales and to derive scores from them, especially in education
+
**Reading: [[Media:Zhu&Simon-1987.pdf|Zhu & Simon paper]]
−
and psychological research:
+
***Posts: Do two posts (total) on the readings.
  
−
  A. [[Media:Harris-article.pdf|Harris Article (PDF)]]
+
=====Multimedia Principles and Cognitive Task Analysis 9-19 to 10-17=====
−
 
+
*9-19 Multi-media Principle
−
  Please take and self-score the test at the end of
+
**Reading: 4.Multi-media Principle (24 pages)
−
  this article.  Count each part of question one as
+
*** Do the quiz and one post.
−
  one point, and each of the remaining three questions
 
−
  as one point (no partial credit!).  Bring your 8
 
−
  scores to class.  E.g. if you missed 1(c) and (d), and
 
−
  you also missed question 4, then you would bring to
 
−
  class the following scores:  
 
−
 
 
−
  1 1 0 0 1 1 1 0
 
−
 
 
−
  If you missed 1(a) and (b) and question 2, bring the
 
−
  following scores:
 
−
 
 
−
  0 0 1 1 1 0 1 1
 
−
 
 
−
  (note that the total score is 5 in both cases, but
 
−
  the pattern of rights and wrongs differs; it is the
 
−
  pattern that we are interested in).
 
−
 
 
−
  B. Please browse *online* through pp 1-23 of the pdf at
 
−
  [http://www.metheval.uni-jena.de/irt/VisualIRT.pdf].
 
−
 
 
−
  The math is a bit heavy going but there are links
 
−
  to apps that illustrate various points in the
 
−
  harris article. 
 
−
 
 
−
  So skim the math and play with the apps.
 
  
−
*3-5
+
*9-24 Empirical CTA: Difficulty Factors Assessment (DFA)
 +
**Reading: [[Media:CogSci97-Heffernan-distrib.pdf‎ | Heffernan paper]]
 +
***Do two posts on the reading
 +
**Come with an attempt at a model of one your task solutions and, ideally, with an initial draft of project step 2.
  
−
The assignment for this lecture has two parts.
+
*9-26 Contiguity Principle
 +
**Reading: 5.Contiguity Principle (24 pages)
 +
**Due: P2:Benchmark Tasks & Rational Cognitive Task Analysis
 +
**Class activity: Peer review of P2
  
−
** (A) An R assignment TBA.  This you can actually email to my by Fri Mar 7.
+
*10-1 From CTA to model building & instructional design
−
** (B) The readings below.
+
**Reading: [[Media:CS02-koedinger-terao-rev.pdf | Picture Algebra paper]]
 +
**Class activity: Work on P3. How will you collect data?
  
−
On Tue we will discuss whatever of A and/or B seem interesting
+
*10-3 Modality Principle
 +
**Reading: 6.Modality Principle (18 pages)
 +
**Class activity: Work on P3. Analyzing your data
  
−
1. "Psychometric Principles in Student Assessment" by Mislevy et al ([[Media:mislevy-principles-2001.pdf|Mislevy (PDF)]])
+
*10-8 Redundancy Principle & CTA via Data Mining
−
   
+
**Reading: 7.Redundancy Principle (18 pages)
−
    Read through p 18.  This is a more modern modern look at some of
+
**Reading: [[Media:Koedinger-et-al-aied2013.pdf | e-learning data to improvement]] (10 pages)
−
    the same issues that are addressed in Trochim's chapters.
 
−
   
 
−
    The remainder of this paper surveys various probabilistic models
 
−
    for the "measurement model" portion of Mislevy's framework (Figure
 
−
    1).  It is quite interesting but we will not pursue it.
 
  
−
2. "Cognitive Assessment Models with Few Assumptions..." by Junker & Sijtsma ([[Media:junker-sijtsma-apm-2001.pdf|Junker, Sijtsma (PDF)]])
+
*10-10 Flex topic: Design & Urban Legends Do Learners Really Know Best? Urban Legends in Education
−
   
+
**Reading: [http://www.articulate.com/rapid-elearning/visual-graphic-design/ Visual & Graphic Design for e-learning blog]
−
    Please read up through p 266 only.
+
**Optional reading: [[Media:Kirschner-Merrienboer-2013.pdf | Do Learners Really Know Best? Urban Legends in Education]]
−
   
+
**Due: P3: Empirical Cognitive Task Analysis & Cognitive Model of Instructional Goals
−
    The math is a bit heavy going so please try to read around it to
 
−
    see what the point of the article is. 
 
−
   
 
−
    We will try to look at some of the data in the article as examples
 
−
    in lecture 2.
 
  
−
*3-7 Continued discussion of Psychometrics [moved Design Research as option for Flex Day]
+
*10-15 Richard Clark visit to class [Was previously Coherence Principle]
 +
**Reading: [[Media:Clark_CTA_In_Healthcare_Chapter_2012.pdf‎ | Clark_CTA_In_Healthcare_Chapter_2012.pdf‎]]
 +
**See his talk on Monday at 3pm in 6115 Gates
 +
**Do 2 posts and come prepared to ask him good questions 
  
−
=====NO CLASS – Spring break 3-12 and 3-14 =====
+
*10-17 Coherence and Personalization Principles
 +
**Reading: 8.Coherence Principle (28 pages)
 +
**Reading: 9.Personalization Principle (26 pages)
  
−
=====Surveys, Questionnaires, Interviews (Kiesler) =====
+
=====Learning By Doing Principles 10-22 to 11-19 =====  
−
* [Plans for these classes were communicated by Kiesler (& Koedinger) via email.]
+
*10-22 Segmenting and Pretraining
−
*3-19
+
**Reading: 10.Segmenting and Pretraining (18 pages)
−
**Reading: Trochim Ch 4 and 5
+
**Do quiz and one post
−
***You already read Ch 5 for the Psychometric section, so just review it.   For both chapters, answer Trochim's on-line questions before and/or after reading (answering the questions before gives you goals for reading).  For discussion board posts, do one post on how have or might use a survey (e.g., of student attitudes) in your own research.  Make another post about Chapter 4, such as something you learned, a question you have, or an answer to someone else's question.
 
−
*3-21
 
−
**Do the following homework assignment [[Media:Arm-modQuestEduc.doc]].  Sara directs: Keep the text that's there and fill in answers, working through it step by step. I'm just as interested in your revisions as in the final version. Est time 45 minutes.
 
−
**Readings
 
−
***Tourangeau, Roger, and T. Yan. 2007. "Sensitive questions in surveys." Psychological Bulletin, 133(5): 859-883.  [[Media:Tourangeau_SensitiveQuestions.pdf]]
 
−
***Tourangeau, R. (2000). “Remembering what happened: Memory errors and survey reports.@ In A. Stone, J. Turkkan, C. Bachrach, J. Jobe, H. Kurtzman, & V. Cain (Eds.), The Science of Self-Report: Implications for research and practice (pp. 29-48). Englewood Cliffs, N.J.: Lawrence Erlbaum.  [[Media:Tourangeau_RememberingWhatHappened.pdf]]
 
  
−
=====Educational Data Mining -- Learning Curve Analysis (Koedinger) =====
+
*10-24 KLI & Selecting appropriate instructional principles
−
*3-26
+
**Reading: KLI sections 6-7
−
**Readings:
+
**DUE: P4: Assessment & Initial Instructional Design
−
***Stamper, J. & Koedinger, K.R. (2011). Human-machine student model discovery and improvement using data. In J. Kay, S. Bull & G. Biswas (Eds.), Proceedings of the 15th International Conference on Artificial Intelligence in Education, pp. 353-360. Berlin: Springer.[[Media:Stamper-Koedinger-AIED2011.pdf| Stamper-Koedinger-AIED2011.pdf]]
+
**Assignment: P5 is due 11-7
−
***'''Optional:'''Ritter, F.E., & Schooler, L. J. (2001). The learning curve.  In W. Kintch, N. Smelser, P. Baltes, (Eds.), International Encyclopedia of the Social and Behavioral Sciences. Oxford, UK: Pergamon. [[Media:RittterSchooler01.pdf | RittterSchooler01.pdf]]
 
−
**'''Assignment:'''  The assignment ([[Media:Learning-curve-assignment-2013_(Geom_Area_Unit_Spring_2010).doc | Learning-curve-assignment-2013.doc]]) is a tutorial on using DataShop to begin analyzing learning curves. (See my emails, original and followup, for further directions on how to do this assignment.)
 
−
*3-28
 
−
**Read the following paper and make two posts on the general topic of this reading and the last, namely, using educational technology data as a basis for discovering improvements to cognitive models.
 
−
***Koedinger, K.R., McLaughlin, E.A., & Stamper, J.C. (2012). Automated student model improvement. In Yacef, K., Zaïane, O., Hershkovitz, H., Yudelson, M., & Stamper, J. (Eds.), Proceedings of the 5th International Conference on Educational Data Mining, pp. 17-24.  [[Media:KoedingerMcLaughlinStamperEDM12.pdf|KoedingerMcLaughlinStamperEDM12.pdf]]
 
−
**Also, do some thinking about a semester project so we can discuss (and I can give feedback) on your possible ideas for a project.
 
−
*4-2
 
−
**Please finish off one of the two exercises you started for last class. See A or B further below. In either case, provide a brief writeup in response to each of the numbered steps and include  a summary of the result you achieved (e.g., did you get a more predictive model as measured by AIC, BIC, or cross validation).  Turn in this writeup and the supporting file (KC model table or R file) on Blackboard.
 
−
**ALSO, make a post about your idea for a course final project.  What method might you apply to address what research question?
 
−
**No required reading assignment.
 
−
***Optional readings:
 
−
***'''Optional:''' Zhang, X., Mostow, J., & Beck, J. E. (2007, July 9). All in the (word) family:  Using learning decomposition to estimate transfer between skills in a Reading Tutor that listens. AIED2007 Educational Data Mining Workshop, Marina del Rey, CA [[Media:AIED2007_EDM_Zhang_ld_transfer.pdf|AIED2007_EDM_Zhang_ld_transfer.pdf]]
 
−
***Roberts, Seth, & Pashler, Harold. (2000). How persuasive is a good fit? A comment on theory testing. Psychological Review, 107(2), 358 - 367. [[Media:2000_roberts_pashler.pdf]]
 
−
***Schunn, C. D., & Wallach, D. (2005). Evaluating goodness-of-fit in comparison of models to data. In W. Tack (Ed.), Psychologie der Kognition: Reden and Vorträge anlässlich der Emeritierung von Werner Tack (pp. 115-154). Saarbrueken, Germany: University of Saarland Press. [[Media:GOF.doc]]
 
  
−
Do A or B:
+
*10-29 Leveraging Examples in E-Learning
−
A. Modify a KC model in a DataShop dataset
+
**Reading: 11.Leveraging Examples in E-Learning (28 pages)
−
1. What is the DataShop dataset you modified?
 
−
2. Describe how you used the HMST procedure (from Stamper paper)
 
−
    to identify a KC to try to improve
 
−
3. Show how you recoded that KC with new KCs (turn in your modified
 
−
    KC file) & describe why you made the change you did
 
−
4. After importing your new KC model to DataShop, did it improve the
 
−
    predictions (are any of the metrics, AIC, BIC, or cross validation)? 
 
−
    (Caution: Make sure your new KC model labels the same number of
 
−
    observations as the KC model you are modifying.)
 
  
−
B. Use R to create an alternative statistical model to AFM
+
*10-31 Does Practice Make Perfect
−
1. Approximate afm in R using either glm or lmer.  How do the parameter
+
**Reading: 12.Does Practice Make Perfect (28 pages)
−
    estimates and metrics (AIC and BIC) compare with results in DataShop?
 
−
2. Modify the regression equation to try to improve the prediction. 
 
−
    Some options include: a) adding a student by KC interaction (there
 
−
    are just main effects of student and KC in AFM), b) adding student
 
−
    slopes (there is just a KC slope in AFM), c) counting success and
 
−
    failure opportunities separately (both kinds of opportunities are
 
−
    lumped together in AFM), d) using log of Opportunity, e) including
 
−
    step (perhaps as a random effect) ...
 
−
3. Turn in your R file including metrics (log-liklihood, parameters,
 
−
    AIC, BIC) on the statistical models you compared
 
−
4. Summarize whether or not your modification changes model fit (log
 
−
    liklihood), changes the number of parameters (from what to what),
 
−
    and, most importantly, improves prediction (as measured by AIC or BIC)
 
  
 +
*11-5 Learning Together Virtually
 +
**Reading: 13.Learning Together Virtually (30 pages)
  
−
=====Educational Data Mining -- Causal Inference from Data (Scheines) =====
+
*11-7 Who’s in Control?
−
*4-4
+
**Reading: 14.Who’s in Control? 30 pages)
−
**Before class on 4-4, do Unit 2 in the OLI course Empirical Research Methods
 
−
go to: http://oli.web.cmu.edu/openlearning/
 
−
in the left tab, go to "Prior work..." and then "Empirical Research Methods"
 
−
click on Peek In
 
−
complete Unit 2
 
−
*4-9
 
−
**Read Scheines, R., Leinhardt, G., Smith, J., and Cho, K. (2005). Replacing lecture with web-based course materials.  Journal of Educational Computing Research, 32, 1, 1-26.  [[Media:Scheines jecr revised.doc | PDF]]
 
−
*4-11 Continue discussion of Causal Inference from Data & TETRAD
 
  
−
===== Flex day (Koedinger) =====
+
*11-12 Simulations and Games
−
*4-16 To be used in case of rescheduling or for a student-driven topic.
+
**Reading: 16.Simulations and Games (32 pages)
−
***And/or for Review of Projects or Past Topics
+
**DUE: P5: Instructional Design Prototyping & Testing
−
**Option1. More on Educational Data Mining
+
**Assignment: P6 is due 11-26
  
−
**Option2. Return to Design Research & Qualitative Methods (Koedinger)
+
*11-14 E-Learning to Build Problem Solving Skill
−
***Trochim Ch 8 (stop before 8.5), Ch 13 (stop before 13.3)
+
**Reading: 15.E-Learning to Build Problem Solving Skill (30 pages)
−
***Barab, S., & Squire, K. (2004). Design-based research: Putting a stake in the ground. The Journal of the Learning Sciences, 13(1).  [[Media:2004 Barab Squire.pdf|PDF]]
 
−
***Optional reading: Chapter on Design Research in Handbook of Learning Sciences
 
  
−
*4-18 NO CLASS - Spring Carnival
+
*11-19 Applying the Guidelines
 +
**Reading: 17.Applying the Guidelines (24 pages)
  
−
=====Experimental Research Methods (Koedinger)=====
+
=====Project Presentations 11-21 to 12-5=====  
−
*4-23
 
−
**Reading: Trochim Ch 7 and 9
 
−
**Do two posts on Blackboard.
 
−
**OLD Slides: [[Media:L02_--_Basic_Research_and_Experimental_Methods.ppt|Experimental_Methods.ppt]] and [[Media:L03-True-Experiments.ppt|True-Experiments.ppt]]
 
−
*4-25 NO CLASS
 
−
*4-30
 
−
**Reading: Trochim Ch 10
 
−
**OLD Slides: [[Media:L04-quasi-experiments.ppt|Quasi-Experiments.ppt]]
 
−
*5-2
 
−
**Reading: Trochim Ch 14
 
−
**Optional:  Try ANOVA module of OLI Statistics course
 
  
−
=====Wrap-up=====
+
*11-21 Project Presentations
−
If needed, schedule a course wrap-up
+
*11-26 Project Presentations
 +
**Faculty course evaluation
 +
**Changed "Due: P6: Research Design" to revise your project with a particular focus on improving steps 3 and 5. Turn all your revisions in as part of the final project and include the reflection statement (see the project assignment handout).
 +
**Assignment: Final Project is due 12-13
 +
*11-28 Thanksgiving, no class
 +
*12-3 Project Presentations
 +
*12-5 Project Presentations
  
−
Final project is due May 10.
+
=====Final Project Due on 12-13=====
 +
*12-13 Project Due

Latest revision as of 15:46, 16 April 2014

Course Details

More recent version at E-Learning Design Principles 2014

Course number & official name: 05-899 Special Topics in HCI: E-Learning Design Principles

Semester: Fall 2013

Carnegie Mellon University

Class times

1:30 to 2:50 Tuesday & Thursday

Location

5222 Gates-Hillman Center (GHC)

Instructor

Professor Ken Koedinger

Office: 3601 Newell-Simon Hall, Phone: 412-268-7667

Email: Koedinger@cmu.edu, Office hours by appointment

Course Prerequisites

To enroll you must either be in the Masters of Educational Technology and Applied Learning Science (METALS) or get the permission of the instruction.

Textbook and Readings

"E-Learning and the Science of Instruction: 3rd edition" by Ruth Colvin Clark and Richard E. Mayer.

Other readings will be assigned in class. See below.

Class URLs

Syllabus and useful links: learnlab.org/research/wiki/index.php/E-learning_Design_Principles_2013

For quizzes and reading reports go www.cmu.edu/blackboard. The course is listed as "Special Topics in HCI".

Goals

This course is about e-learning design principles, the evidence and theory behind them, and how to apply these principles to develop effective educational technologies. It is organized around the book "e-Learning and the Science of Instruction: Proven Guidelines for Consumers and Designers of Multimedia Learning" by Clark & Mayer with further readings drawn from cognitive science, educational psychology, and human-computer interaction. You will learn design principles 1) for combining words, audio, and graphics in multimedia instruction, 2) for combining examples, explanations, practice and feedback in online support for learning by doing, and 3) for balancing learner versus system control and supporting student metacognition. You will read about the experiments that support these design principles, see examples of how to design such experiments, and practice applying the principles in educational technology development.

Flipped Homework: Reading Reports and Reading Quizzes

You will have "flipped homework", a variation on the flipped classroom idea you might have heard of. Flipped homework is an assignment before a relevant class meeting rather than after it. It helps you to check your understanding of what you read, to practice to enhance your memory (we will talk about the "testing effect" in class), and to get a better sense of what you don't know so you are prepared to ask questions in class. It also helps instructors focus the class discussion to better avoid belaboring known points and pursue student needs and interests.

Before some class sessions, you will asked to do a quiz associated with the assigned book chapter. The quizzes will be on the Blackboard site (www.cmu.edu/blackboard, the course is listed as "Special Topics in HCI"). Before other class sessions, you will be asked to write "reading reports". We will use the discussion board on Blackboard. You should complete assigned quizzes or reading reports before 9am' on the day of class.

For reading reports, the discussion forum post will usually direct you as to how to reply. If not otherwise directed, you should make two posts on the readings. Your two posts may be original or in response to another post (one of both is nice).

  • Original posts should contain one or more of the following:
    • something you learned from the reading or slides
    • a question you have about the reading or slides or about the topic in general
    • a connection with something you learned or did previously in this or another course, or in other professional work or research
  • Replies should be an on-topic, relevant response, clarification, or further comment on another student’s post.

In general, please come to class prepared to ask questions and give answers.

Grading

  • 55% Final Project
    • 45% Six parts of final project, 7.5% each
    • 10% Final project submission
  • 5% E-Learning examples assignment
  • 10% Peer review and feedback
  • 10% Pre-class quizzes & reading reports
  • 10% Chapter summary
  • 10% Class participation

Class Schedule in Brief

  • E-Learning Introduction 8-27 to 9-5
    • Aug 27 Overview; Examples Assignment; Project
    • Aug 29 1.E-learning; KLI Framework events (The "1." indicates this is a chapter in the Clark & Mayer book)
    • Sept 3 2.How People Learn; KLI KC's
    • Sept 5 3.Evidence-based practice; KLI Learning & Instructional Events
  • Instructional Goals and Cognitive Task Analysis 9-10 to 9-17
    • Sept 10 Determining instructional goals (tasks)
    • Sept 12 Guest lecture
    • Sept 17 Discovering learning objectives (KCs) & Rational Cognitive Task Analysis
  • Multimedia Principles and Cognitive Task Analysis 9-19 to 10-17
    • Sept 19 4.Multi-media Principle
    • Sept 24 Empirical Cognitive Task Analysis: Think aloud
    • Sept 26 5.Contiguity Principle
    • Oct 1 CTA: DFA & Model building
    • Oct 3 6.Modality Principle & 7.Redundancy Principle
    • Oct 8 CTA & Designing Assessments for Continual Improvement
    • Oct 10 Midterm review; Flex topic
    • Oct 15 8.Coherence Principle
    • Oct 17 9.Personalization Principle
  • Learning By Doing Principles 10-22 to 11-19
    • Oct 22 10.Segmenting and Pretraining
    • Oct 24 KLI & Selecting appropriate instructional principles
    • Oct 29 11.Leveraging Examples in E-Learning
    • Oct 31 12.Does Practice Make Perfect
    • Nov 5 13.Learning Together Virtually
    • Nov 7 14.Who’s in Control?
    • Nov 12 15.E-Learning to Build Problem Solving Skill
    • Nov 14 16.Simulations and Games
    • Nov 19 17.Applying the Guidelines
  • Project Presentations 11-21 to 12-5
    • Nov 21 Project Presentations
    • Nov 26 Project Presentations
    • Nov 28 Thanksgiving, no class
    • Dec 3 Project Presentations
    • Dec 5 Project Presentations
  • Final Project due Dec 13

Class Schedule with Readings and Assignments

NOTE: This section is "living" -- it will grow and change as the semester goes on.

E-Learning Introduction 8-27 to 9-5
  • 8-27 Overview, course project, your interests
    • Class activity: Discuss your interests in e-learning
    • Assignment: Examples (click to get) is due next Thursday, 9-5
      • BRING two screen shots of your first example to next class
    • Assignment: Project step 1 is due in 16 days on Thursday, 9-12
    • NOTE: See reading assignment for next time on next date.
  • 8-29 E-learning intro and KLI Framework events (Click here for slides)
    • Class activity: Promises & pitfalls review of e-learning examples
      • BRING two screen shots of your first example to this class
    • Reading (from course book): 1.e-Learning: Promise & Pitfalls (28 pages). You can get this chapter here this time but order the book right now!
      • Pre-class quiz: Answer questions for Chpt1 Quiz on Blackboard
    • For next time:
      • BRING two screen shots of your second example to this class
      • Review project step1 and come with a preliminary project idea. You might write some thoughts down, but you do not need to hand anything in.
      • a) Do the two readings, b) associated quiz & c) discussion board post on Blackboard
  • 9-3 How People Learn and KLI Knowledge Components (Slides)
    • Read Ch2.How Do People Learn from E-Courses (20 pages) You can get this chapter here this last time!
      • Pre-class quiz: Answer questions for Chpt2 Quiz on Blackboard (5 minutes)
    • Read KLI Framework paper sections 1-3 (18 pages)
      • Make one post to Blackboard -- see questions in Forum introduction
    • Class activity: KC type in e-learning examples
      • BRING two screen shots of your second example to this class.
    • Class activity: Project idea discussion
      • Come prepared with a preliminary project idea
  • 9-5 Evidence-based practice and KLI Learning & Instructional Events Media: (Slides)
    • Reading: 3.Evidence-based practice (18 pages)
      • Pre-class quiz: Answer questions for Chpt3 Quiz on Blackboard
    • Reading: KLI sect 4-5 (12 pages)
      • Make one post to Blackboard -- see questions in Forum introduction
    • Class activity: Principles present in e-learning examples
    • DUE: Examples assignment is due at beginning of class. Please submit on blackboard.
Instructional Goals and Cognitive Task Analysis 9-10 to 9-17
  • 9-10 Goals, assessment tasks, cognitive task analysis, and instructional design
    • Class activity: Review Project ideas and step 1 write-up requirements; consider assessment tasks
    • Reading: Feldon paper
      • Posts: Do two posts on the Feldon reading.
  • 9-12 Cognitive Task Analysis and Think Alouds by guest lecturer Vincent Aleven
    • DUE: Project step P1: Domain, Context & Initial Resources
    • Assignment: Project step P2 is due on 9-26
    • Reading: Lovett paper and Gomoll paper
      • Posts: Do two posts (total) on the Lovett and Gomoll papers.
  • 9-17 Discovering learning objectives (KCs) and Rational Cognitive Task Analysis
Multimedia Principles and Cognitive Task Analysis 9-19 to 10-17
  • 9-19 Multi-media Principle
    • Reading: 4.Multi-media Principle (24 pages)
      • Do the quiz and one post.
  • 9-24 Empirical CTA: Difficulty Factors Assessment (DFA)
    • Reading: Heffernan paper
      • Do two posts on the reading
    • Come with an attempt at a model of one your task solutions and, ideally, with an initial draft of project step 2.
  • 9-26 Contiguity Principle
    • Reading: 5.Contiguity Principle (24 pages)
    • Due: P2:Benchmark Tasks & Rational Cognitive Task Analysis
    • Class activity: Peer review of P2
  • 10-1 From CTA to model building & instructional design
  • 10-3 Modality Principle
    • Reading: 6.Modality Principle (18 pages)
    • Class activity: Work on P3. Analyzing your data
  • 10-15 Richard Clark visit to class [Was previously Coherence Principle]
  • 10-17 Coherence and Personalization Principles
    • Reading: 8.Coherence Principle (28 pages)
    • Reading: 9.Personalization Principle (26 pages)
Learning By Doing Principles 10-22 to 11-19
  • 10-22 Segmenting and Pretraining
    • Reading: 10.Segmenting and Pretraining (18 pages)
    • Do quiz and one post
  • 10-24 KLI & Selecting appropriate instructional principles
    • Reading: KLI sections 6-7
    • DUE: P4: Assessment & Initial Instructional Design
    • Assignment: P5 is due 11-7
  • 10-29 Leveraging Examples in E-Learning
    • Reading: 11.Leveraging Examples in E-Learning (28 pages)
  • 10-31 Does Practice Make Perfect
    • Reading: 12.Does Practice Make Perfect (28 pages)
  • 11-5 Learning Together Virtually
    • Reading: 13.Learning Together Virtually (30 pages)
  • 11-7 Who’s in Control?
    • Reading: 14.Who’s in Control? 30 pages)
  • 11-12 Simulations and Games
    • Reading: 16.Simulations and Games (32 pages)
    • DUE: P5: Instructional Design Prototyping & Testing
    • Assignment: P6 is due 11-26
  • 11-14 E-Learning to Build Problem Solving Skill
    • Reading: 15.E-Learning to Build Problem Solving Skill (30 pages)
  • 11-19 Applying the Guidelines
    • Reading: 17.Applying the Guidelines (24 pages)
Project Presentations 11-21 to 12-5
  • 11-21 Project Presentations
  • 11-26 Project Presentations
    • Faculty course evaluation
    • Changed "Due: P6: Research Design" to revise your project with a particular focus on improving steps 3 and 5. Turn all your revisions in as part of the final project and include the reflection statement (see the project assignment handout).
    • Assignment: Final Project is due 12-13
  • 11-28 Thanksgiving, no class
  • 12-3 Project Presentations
  • 12-5 Project Presentations
Final Project Due on 12-13
  • 12-13 Project Due
Retrieved from "https://learnlab.org/wiki/index.php?title=E-learning_Design_Principles_2013&oldid=12808"