Difference between revisions of "Educational Research Methods 10"

From LearnLab
Jump to: navigation, search
(→‎Video and Verbal Protocol Analysis (Lovett, Rosé))
Line 170: Line 170:
 
**Kumar, R., Ai, H., and Rosé (submitted).  Choosing Optimal Levels of Social Interaction – Towards creating Human-like Conversational Tutors, submitted to the Intelligent Tutoring Systems Conference[[Media:Its2010-thermo-rk.pdf|ITS2010-Kumar]]
 
**Kumar, R., Ai, H., and Rosé (submitted).  Choosing Optimal Levels of Social Interaction – Towards creating Human-like Conversational Tutors, submitted to the Intelligent Tutoring Systems Conference[[Media:Its2010-thermo-rk.pdf|ITS2010-Kumar]]
 
** Data set from Kumar et al. study [[Media:Thermo-f09-transcripts.xls‎|Data]]
 
** Data set from Kumar et al. study [[Media:Thermo-f09-transcripts.xls‎|Data]]
 +
** Iris's coding manual [[Media:Thermo08CodingScheme-v9b.doc|Manual]]
 +
** Chapter with alternative presentation of Reasoning coding [[Media::Sionti-Revised-Again.doc|Chapter]]
 
*2-16-10
 
*2-16-10
−
** Rosé, C. P., Wang, Y.C., Cui, Y., Arguello, J., Stegmann, K., Weinberger, A., Fischer, F., (In Press).  Analyzing Collaborative Learning Processes Automatically: Exploiting the Advances of Computational Linguistics in Computer-Supported Collaborative Learning, submitted to the International Journal of Computer Supported Collaborative Learning [[http://www.cs.cmu.edu/~cprose/pubweb/Proof2.pdf]]
+
** Rosé, C. P., Wang, Y.C., Cui, Y., Arguello, J., Stegmann, K., Weinberger, A., Fischer, F., (2008).  Analyzing Collaborative Learning Processes Automatically: Exploiting the Advances of Computational Linguistics in Computer-Supported Collaborative Learning, submitted to the International Journal of Computer Supported Collaborative Learning [[http://www.cs.cmu.edu/~cprose/pubweb/Proof2.pdf]]
 
** Ai, H., Kumar, R., Nagasunder, A., Rose, C. P. (submitted).  Exploring the Effectiveness of Social Capabilities and Goal Alignment in Computer Supported Collaborative Learning, submitted to the Intelligent Tutoring Systems Conference[[Media:Its2010_submission_172.pdf]]‎  
 
** Ai, H., Kumar, R., Nagasunder, A., Rose, C. P. (submitted).  Exploring the Effectiveness of Social Capabilities and Goal Alignment in Computer Supported Collaborative Learning, submitted to the Intelligent Tutoring Systems Conference[[Media:Its2010_submission_172.pdf]]‎  
 
*2-18-10
 
*2-18-10

Revision as of 19:45, 10 February 2010

Research Methods for the Learning Sciences 85-748

Spring 2010 Syllabus Carnegie Mellon University

Class times

4:30 to 5:50 Tuesday & Thursday

Location

336B Baker Hall for the first day.

3501 Newell Simon Hall thereafter.

Instructors

Professor Ken Koedinger

Location: 3601 Newell-Simon Hall

Phone: 8-7667

Email: Koedinger@cmu.edu

Office hours by appointment


Dr. Philip I. Pavlik Jr.

Location: 300S Craig St, 224

Phone: 8-1618

Email: ppavlik@andrew.cmu.edu

Office hours by appointment

Teaching Assistant

Benjamin Shih

Location: GHC 8003

Phone: 8-6289

Email: shih@cmu.edu

Office hours by appointment

Class URL

learnlab.org/research/wiki/index.php/Educational_Research_Methods_10

Goals

The goals of this course are to learn data collection, design, and analysis methodologies that are particularly useful for scientific research in education. The course will be organized in modules addressing particular topics including overview of methods, cognitive task analysis, qualitative methods, protocol and discourse analysis, and educational data mining and log analysis. A key goal is to help students think about and learn how to apply these methods to their own research programs.

Course Prerequisites

To enroll you must have taken 85-738, "Educational Goals, Instruction, and Assessment" or get the permission of the instruction.

Textbook and Readings

"The Research Methods Knowledge Base: 3rd edition" by William M.K. Trochim and James P. Donnelly. You can find it at www.atomicdogpublishing.com/BookDetails.asp?BookEditionID=160

Other readings will be assigned in class.

Reading Reports

We will be using Google Wave for course reading reports and discussions. Google Wave combines discussion boards, instant messengers, and wikis into a single system. You can use it just as you would a discussion board, but you can also edit your own / other peoples' posts, play back the changes, and see changes update in real-time. Further details and account invitations will be discussed in class.

Reading reports consist of three parts: students are required to submit at least one original post per reading assignment, at least one reply or comment on another student's post, and at least one substantive addition to the reading assignment summary. More posts, replies, and summary improvements are encouraged.

Monday Tuesday Wednesday Thursday Friday Saturday Sunday
Original Post (Tuesday Reading) Reply (Tuesday Reading)

Summary Edits (Tuesday Reading)

Original Post (Thursday Reading)

Summary Edits (Thursday Reading)

Reply (Thursday Reading)
Posts and Replies

Original posts should contain at least one of the following:

  • a question you had about the reading or something important you did not understand
  • an idea inspired by the reading
  • an interesting connection with something you learned or did previously in this or another course, or in other professional work or research

For readings due on a Tuesday, the original post must be submitted by Monday morning.

For readings due on a Thursday, the original post must be submitted by Thursday morning.

Replies must be:

  • an on-topic, relevant response, clarification, or further comment on another student’s post

For readings due on a Tuesday, at least one reply must be submitted by Tuesday morning.

For readings due on a Thursday, at least one reply must be submitted by Sunday morning.

This means that replies for Tuesday readings are due before class, whereas replies for Thursday readings are due after. Please use this extra time to have a full and meaningful discussion on the topics discussed.

Summary

For each reading assignment, one student will be responsible for a finished summary of that assignment and its related discussion.

Each summary will consist of:

  • A brief overview of the reading assignment. For a chapter from the textbook, this should be a couple sentences on major topics addressed in the chapter. For a research paper, this should be a couple sentences covering the research question(s) and primary result(s).
  • A brief discussion of the methodology. For a chapter from the textbook, this should be a more detailed discussion of the main research methodology discussed. For a research paper, this should be a couple sentences discussing aspects of the data, such as the subject population or analytical methods.
  • A listing of major issues or suggestions for the paper, as related to the course. Threats to validity and problems with test reliability are example topics, as well as suggestions on how to avoid or resolve such issues.

The first two parts of the summary should be complete by the morning of the day of class.

Grading

There will be assignments associated with each section of the course. Grades will be determined by your performance on these assignments, by your participation in Reading Reports, and by your participation in class.

  • Course work
    • 10% Reading reports
    • 50% Homework assignments
  • Project & final paper
    • 40% Design a new study based on one (or more) of these methods that pushes your own research in a new direction.

Class Schedule

Basic Research & Experimental Methods (Koedinger, Pavlik)
Cognitive Task Analysis (Koedinger, Pavlik)
  • 1-26-10
    • Clark, R. E., Feldon, D., van Merriënboer, J., Yates, K., & Early, S. (2007). Cognitive task analysis: In J. M. Spector, M. D. Merrill, J. J. G. van Merriënboer, & M. P. Driscoll (Eds.), Handbook of research on educational communications and technology (3rd ed., pp. 577–593). Mahwah, NJ: Lawrence Erlbaum Associates.
    • Assignment 2: Assignment2.doc
    • Slides: CTA-01.pdf
  • 1-28-10
    • Rittle-Johnson, B. & Koedinger, K. R. (2001). Using cognitive models to guide instructional design: The case of fraction division: In Proceedings of the Twenty-Third Annual Conference of the Cognitive Science Society, (pp. 857-862). Mahwah,NJ: Erlbaum.
    • Heffernan, N. & Koedinger, K. R. (1997). The composition effect in symbolizing: The role of symbol production vs. text comprehension: In Shafto, M. G. & Langley, P. (Eds.) Proceedings of the Nineteenth Annual Conference of the Cognitive Science Society, (pp. 307-312). Hillsdale, NJ: Erlbaum.
Video and Verbal Protocol Analysis (Lovett, Rosé)
  • 2-4-10: In this introductory lecture, we will discuss the main steps of protocol analysis and what can be gained from the process. We will discuss these 2 readings in class.
    • Ericsson, K. A., & Simon, H. A. (1993). Protocol Analysis: Verbal Reports as Data (Revised Edition, pp. xii-xv). Cambridge, MA: MIT Press. Media:E&SPreface.pdf
    • Gilhooly, K. J., Fioratou, E., Anthony, S. H., & Wynn, V. (2007). Divergent thinking: Strategies and executive involvement in generating novel uses for familiar objects, British Journal of Psychology, 98, 611-625. Media:Gilhooly.pdf‎
  • 2-9-10 Protocol Analysis of Educational Discussions
    • [Half the class will read this one]Veel, R. (1999). Language, knowledge and authority in school mathematics, in Francis Christie (Ed.) Pedagogy and the Shaping of Consciousness: Linguistics and Social Processes, Continuum. Media:PedagogyChapter_7.pdf
    • [Half the class will read this one] Williams, G. (1999). The pedagogic device and the production of pedagogic discourse: a case example in early literacy education, in Francis Christie (Ed.) Pedagogy and the Shaping of Consciousness: Linguistics and Social Processes, Continuum. Media:PedagogyChapter_4.pdf
    • [Optional] Martin, J. and White, P. R. (2005). The Language of Evaluation: Appraisal in English, Chapter 3, Palgrave. Media:Martin-WhiteChapter_3.pdf
  • 2-11-10
    • van Someren, M. W., Barnard, Y. F., & Sandberg, J. A. C. (1994).The Think Aloud Method: A Practical Guide to Modelling Cognitive Processes. New York: Academic Press. Chapter 7Media:VanSch7.pdf‎
    • Kumar, R., Ai, H., and Rosé (submitted). Choosing Optimal Levels of Social Interaction – Towards creating Human-like Conversational Tutors, submitted to the Intelligent Tutoring Systems ConferenceITS2010-Kumar
    • Data set from Kumar et al. study Data
    • Iris's coding manual Manual
    • Chapter with alternative presentation of Reasoning coding [[Media::Sionti-Revised-Again.doc|Chapter]]
  • 2-16-10
    • Rosé, C. P., Wang, Y.C., Cui, Y., Arguello, J., Stegmann, K., Weinberger, A., Fischer, F., (2008). Analyzing Collaborative Learning Processes Automatically: Exploiting the Advances of Computational Linguistics in Computer-Supported Collaborative Learning, submitted to the International Journal of Computer Supported Collaborative Learning [[1]]
    • Ai, H., Kumar, R., Nagasunder, A., Rose, C. P. (submitted). Exploring the Effectiveness of Social Capabilities and Goal Alignment in Computer Supported Collaborative Learning, submitted to the Intelligent Tutoring Systems ConferenceMedia:Its2010_submission_172.pdf‎
  • 2-18-10
    • Schooler, J. W., Ohlsson, S., & Brooks, K. (1993). Thoughts Beyond Words: When Language Overshadows Insight, Journal of Experimental Psychology 122(2), pp 166-183. Media:Schooleretal.pdf‎
  • 2-23-10
    • Download SIDE and the SIDE User's Manual from the webpage. [[2]]
Psychometrics, reliability, Item Response Theory (Junker, Koedinger)
  • 2-25-10
  • 3-2-10
  • 3-4-10
NO CLASS – Spring break
  • 3-9-10
  • 3-11-10
OPEN? [Design Research?]
  • 3-16-10
Surveys, Questionnaires, Interviews (Kiesler)
  • 3-18-10
  • 3-23-10
Educational data mining (Scheines, Pavlik, Koedinger)
  • 3-25-10
  • 3-30-10
  • 4-1-10
  • 4-6-10
  • 4-8-10
  • 4-13-10
  • 4-15-10 NO CLASS – Spring Carnival
Cognitive Task Analysis - Revisited (Koedinger, Pavlik)
  • 4-20-10
  • 4-22-10
Wrap-up
  • 4-27-10
  • 4-29-10
Retrieved from "https://learnlab.org/wiki/index.php?title=Educational_Research_Methods_10&oldid=10591"