Assessment is such an important topic with regard to MOOCs and other new learning models that I plan to devote a week to the subject in April. But for now, I’d like to provide an argument why testing is both the greatest vulnerability for moving MOOCs forward, as well as the ripest area for innovation in online learning.
To begin with, if you run down the list of course components discussed this week (which so far covered lectures, homework, reading, and discussion/community) very few of these require complete re-imagination when a course moves from the physical classroom to the Web.
Lectures (typically 50-90 minutes long in a traditional class) are already broken down into smaller increments in most virtual courses (from the 1-2 minute micro-lectures found in the Udacity classes I’ve enrolled in, to strings of linked 5-15 lectures that are the preferred model for courses I’ve taken from Coursera and EdX, to the strict 30-minute standard of Great Courses audio classes).
Theoretically, lectures might eventually be split still further and fit into some sort of adaptive branching model. But, as the leaders of both Coursera and EdX pointed out at an MIT meeting I attended last night, these options present technical and pedagogical challenges that may make them less beneficial than one might think in terms of delivering learning.
But regardless of how you slice, dice and arrange the lecture, the best courses will always involve an expert with talented presentation skills communicating information to students (whether face-to-face or via screen or iPod). While some courses I’ve taken separate from this project eliminate lectures entirely in favor of guided reading and online communication, these do not reflect the model for the most compelling multi-media options currently generating such excitement (and concern) within the educational community.
And speaking of reading, as far as I can tell this still involves students absorbing text-based materials (whether delivered to them on paper or on screen). The notion of how much reading should be associated with a college-level course is an important point to debate, particularly as we evaluate whether a MOOC or other online experience is truly as rigorous as its’ classroom equivalent. And such a conversation should include whether one mission of the college course is to enforce reading of a certain amount or certain set of texts.
But these are debates over quantity, not over the qualitative nature of a learning component (in this case reading). Similarly, even the technological options supporting online equivalents of classroom discussion (including online discussion forums, social networks and audio/video conferencing technologies) mostly bring up issues relating to numbers (i.e., how to get enough students to contribute and interact enough without swamping online forums to the point where they become unusable).
But testing – or any technique for evaluation (embodied in homework, quizzes, exams, graded writing assignments or class projects) is something that can’t be simply moved from a professor’s paper file to the screen without thinking through issues such as:
- What is the purpose behind a specific evaluation exercise (be it homework, a quiz or high-stakes exam)? Is it to ensure comprehension of a reading assignment or lecture point? Measure recall of factual knowledge? Give students the ability to demonstrate critical thinking or other higher-order skills?
- What assessment techniques are practical in the context of an online class delivered to large numbers of students?
- How do we know whether those chosen assessments are actually measuring what we want measured?
One advantage of the traditional classroom is that there exists an authority (the professor) who acts as the setter of standards. And regardless of whether this authority is perfectly consistent or wildly arbitrary in how he or she grades assignments (or asks his or her TAs to do perform such grading), the dozens (or, in some cases, hundreds) of students enrolled in a college class know what they’re getting into when they select a teacher.
But once grades are assigned based on machine-scored multiple-choice quizzes or peer-reviewed writing assignments (the two most popular assessment techniques I’ve seen so far in the MOOC classes I’ve taken), to whom can a student turn if they feel that a test question stinks (or is wrong) or if they don’t feel their peers adequately graded their work?
And if students don’t feel that this week’s assessments are being handled rigorously, how seriously will they take next week’s? Finally, if it’s an open question whether material taught has actually been learned, how strong an argument can we really make regarding the comparability between a MOOC and a brick-and-mortar college class?
Fortunately, many problems related to how to give consistent testing to large diverse populations have been addressed (if not entirely solved) within industries such as the standardized educational testing, professional licensure and certification. And in a few weeks time, we’ll take a look at how we might be able to apply some of the science and solutions they’ve been working on to this vital aspect of the online learning experience.