Quantitative and Experimental Methods for Designing and Evaluating Learning
Start Any Time
Work on your pace and you will have instructors available to help you answer any questions.
Duration
Approximately 3 weeks, 6-8 hours/week
Fee
$1500 Professional Rate
$500 Full-time Student Rate
*Proof of full time student enrollment required. Acceptable forms of id include a letter from your university’s registrar office or an unofficial transcript. Email your documents to learnlab-help@lists.andrew.cmu.edu
Certificate Course Description:
In this introductory course, you will learn about to quantitative cognitive task analysis methods and experimentation in e-learning. Successfully completing this course will set you on the path to becoming an expert in quantitative and experimental learning engineering. If you find it interesting and want to continue learning more about this topic, you may continue to explore it more detail by taking other courses that elaborate on the principles and methods used in learning engineering.
Our team hosts office hours for all courses over Zoom on the 2nd and 4th Wednesday of each month, 10:00 AM – 11:00 AM EDT/EST (4:00 PM – 5:00 PM CAT). On 2nd Wednesdays, Dr. Ken Koedinger, who is the Director of the Masters of Educational Technology and Applied Learning Sciences (METALS) program, will be available. On 4th Wednesdays, you will have the opportunity to speak with our learning engineers.
Module 1: Quantitative Cognitive Task Analysis: Difficulty Factors Assessment
- Perform & explain the steps in a Difficulty Factors Assessment (DFA)
- Given a task, identify a difficulty factor in it and design a matched task without that difficulty
- Create assessment forms; analyze errors
- Model results and use to design instruction
- Explain how the DFA steps implement the E-learning Big Picture
- Compare and contrast Difficulty Factors with prior CTA techniques
- Distinguish between two kinds of empirical CTA
Module 2: Quantitative CTA via Data Mining
- Explain how student interactions in an e-learning system can be used to perform Cognitive Task Analysis
- Recognize the features of a good learning curve
- Use learning curves to do Cognitive Task Analysis
- Recognize a “bad” learning curve
- Analyze tasks to hypothesize new KCs to improve curve
- Evaluate new KC model in comparison to old
- Use DataShop tools to aid these steps
- Evaluate and redesign a tutor based on improved KC model and associated insights
Module 3: In vivo experimentation; A/B testing
- Identify the characteristics of a good experiment
- Explain why optimizing near termperformance (e.g., through better UX design) does not guarantee optimal learning
- Explain why experimentation is important and illustrate with cases where intuition and theory are inadequate
- Design a learning experiment (or A/B test)to evaluate whether one e-learning design is better than a close alternative
- Explain ethical considerations for running A/B testing studies in education
Module 5: Course Project or Final Exam
At the end of the course, you’ll have an opportunity to do a little project where you will analyze learning curves using a tool developed at CMU called DataShop. That will provide you with a nice experience to apply the fundamentals you will learn in the modules to a larger, more authentic, context. It will be graded by the instructor and you will receive personalized feedback along with a sample solution.
You will have an alternative option to take a final exam where you will answer 20 questions. The exam can be taken multiple times and each time new questions are randomly selected from a pool of more than 150 questions.
You are also free to do both the course project and the final exam, we will consider the one in which you score more for counting towards the certificate.
We recommend knowledge of or experience with cognitive task analysis or this course: Uncovering Implicit Knowledge with Cognitive Task Analysis
Researchers, product/UX designers, instructional designers, and students who want to learn about various techniques for enriching active learning. Anyone interested in edtech.
What you'll learn
This course will help you:
- Perform & explain the steps in a Difficulty Factors Assessment (DFA)
- Evaluate and redesign an e-learning design through Cognitive Task Analysis using learning curves
- Recognize a “bad” learning curve
- Analyze tasks to hypothesize new Knowledge Components to improve curve
- Evaluate new Knowledge Component model in comparison to old
- Use DataShop tools to aid these steps
- Design a learning experiment (or A/B test) to evaluate whether one e-learning design is better than a close alternative
Course Instructors
Dr. Ken Koedinger
is a professor of Human-Computer Interaction and Psychology at Carnegie Mellon University. Dr. Koedinger has an M.S. in Computer Science, a Ph.D. in Cognitive Psychology, and experience teaching in an urban high school. His multidisciplinary background supports his research goals of understanding human learning and creating educational technologies…
Certificate
Upon successful completion of the program, participants will receive a verified digital certificate of completion from Carnegie Mellon University’s Open Learning Initiative.
In addition to the knowledge and immediately applicable frameworks you will gain by attending your selected courses, you will benefit from:
- A digital, verified version of your Executive Certificate (Smart Certificate) you can add to your resume and LinkedIn
- Networking with a global group of your peers and instructors for advancing your career
Register Now
Register and start taking the course in four steps:
1. Enter your email address
2. Watch this short video for instructions on how to register in OLI.
3. For this course, copy the course key: QEMDEL-001
4. Click on this link to Carnegie Mellon University’s Open Initiative to register and try out the course for 48 hours before payment is due.
5. (Optional but highly recommended) Set OLI to automatically resume from where you left off in the course.
1) Click on your name in the upper right corner to bring up your OLI profile settings.
2) Change the option on the dropdown to resume automatically.
3) Lastly, select UPDATE.
You are all set!
