If I had been less of a wuss and chosen to drive through the snow to last Tuesday’s presentation of research findings from HarvardX, and had I made it and been fortunate enough to get to ask a question, the one I would have raised would have been about the results of one specific course – CopyrightX – and what they might say about MOOCs overall.
Of the many innovations this course included, and the one that makes it most interesting from the broader MOOC perspective, was enrollment limited to just 500 students. And these 500 students were selected from a pool of over 4000 who demonstrated their interest in the course by filling out a pretty robust application which included, among other things, three short essays.
Many of the course materials (including video lectures, reading assignments and lecture notes) were provided to anyone interested in just auditing the course. But the 500 students who made it through an application process with roughly the same acceptance rate of a traditional high-end college were also asked to participate in synchronous “Socratic discussions” covering specific case studies, attend “special event” lectures by guests talking about copyright issues in fields like fashion and software, and pass a take-home final exam that would be graded by live teaching assistants.
The combination of selective, limited enrollment, live discussion and grades based on attendance/participation and passing of a rigorous, non-machine-scored exam brought CopyrightX much more in line with the structure of a traditional (albeit large) brick-and-mortar or online class. Which might make you think that completion and certification rates for this course should far surpass those of standard MOOCs from Harvard and elsewhere.
So what did the actual numbers say?
Fortunately, one of those research reports that were released on Tuesday, one that covers CopyrightX, is publically available and can be read in full here. And, keeping in mind that certification required participation in most of the Socratic discussion sessions and passing the final exam, 55.4% of enrollees made it to the final discussion group meeting, 61.4% attended enough discussion sessions to pass that requirement, 247 students (about half) took the take-home final, of which 195 passed. And taking the two certification requirements (participation in discussion and passing the final exam) together, a total of 193 students (or 38.6%) earned a certificate.
At first glance, these numbers indicate a certification rate that far exceeds the percentage of enrollees who earn a certificate in any open-enrollment MOOC course (usually less than 10%). But as noted previously, students hitting the Enroll button on the edX or other MOOC web site tend to spread themselves out over a continuum of activity, with browsers usually never getting past initial registration (and maybe viewing 1-2 weeks of material), “Explorers” (or auditors – who got separated out through the application process in CopyrightX) watching most of the lecture videos who are not necessarily interested in taking graded assignments, and certified students who do the work necessary to pass the course and earn a certificate (which doesn’t necessarily correspond to watching all the lectures and doing all class assignments – beyond what is needed to earn a certificate).
In fact, of students who indicate their interest in taking an unlimited enrollment MOOC to completion (by finishing the first graded assignment), certification rates are closer to 40%, which is actually a bit higher than the overall certification rate in CopyrightX.
Now unlike most MOOC courses with relatively low pass/fail scores (which makes it highly likely that any dedicated student who has put effort into learning the material will earn a certificate), CopyrightX had a tough final passed by less than 80% of the students who took it. So a better number to compare to traditional MOOC completion rates would be the 50% who took the final (ignoring the small percentage that took that exam but didn’t take part in enough discussion to pass the course).
I think it’s fair to say that fifty percent going all the way in a class with highly demanding participation and exam requirements is significant when compared to the 40% finishing rate of students who demonstrated a commitment to completing the less-demanding requirements of an open-enrollment MOOC. But given CopyrightX’s stiff application threshold (which, as noted earlier, is akin to applying to college) one would think completion rates would be closer to the 80-90% you find in traditional or online for-credit courses.
Which brings this whole discussion back to the economic motivation arguments that have come up before in MOOC discussions. For it seems that asking students to apply for a limited enrollment MOOC provides enough incentive to get 50% vs. 40% of students to do all the work (continuing to take into account that work in this particular limited enrollment course is highly demanding). But it doesn’t give them enough “skin in the game” (boy do I hate that phrase) as has someone who applied to take – and paid to take – a for-credit course (either classroom or online).
One of the challenges in drawing this or any other conclusion based just on course data is the enormous variability from MOOC to MOOC. CopyrightX’s required simultaneous online discussion and an expert graded (not peer-graded) written final exam, features that were certainly unique when compared to other MOOCs I completed last year. But other courses I took (such as Cheating and ChinaX) included guest lecturers. And while the one piece of written work in CopyrightX looked quite demanding (it’s provided in the research results report), the eight papers I wrote for Coursera’s The Modern and the Postmodern made up for difficulty on each assignment with quantity of assignments.
On top of that, one of the other important lessons I learned last year is that a fantastic lecturer is a fantastic lecturer whose courses delivered the goods (i.e., education) with or without technical bells and whistles and even (at least for students whose primary motivation was learning) without challenging pass/fail requirements.
In fact, it is the sheer variability of courses (something encouraged by the culture of experimentation that is at the heart of the MOOC project) that makes it difficult to base understanding on statistics such as average scores and pass rates.
But the variability of these objects of study (MOOC courses) is as nothing when compared to the variability of those who provide the data for those studies (MOOC students) a wild and wooly group (of which I am a member) that I plan to take a look at next time.
Leave a Reply