Skip to content

What is effective assessment in a MOOC?

Le Penseur. The Thinker.
Le Penseur. The Thinker.
By: Sigfrid Lundberg.

Recently I completed my first Massive Open Online Course (MOOC). Which one is not relevant to this post, as I don't intend this to be a review of the specific course. After completing the course, I started to reflect on the way assessment was used. These reflections led me to the conclusion that I don't believe it was particularly effective. This conclusion was quickly followed by another, I do not know what the alternatives are.

Let me explain...

The course was structured around six weeks of video based content. Each week also included three different types of assessment; a wiki, a discussion forum and a quiz. I will explore my reflections on each of these assessment types below.

Wikis: A search for more information

The wiki was used to encourage the students to search out more information related to the content presented that week. For example, additional references, the organisations mentioned, the reports mentioned, or definitions of key terms.

The Wiki used Markdown, a popular lightweight markup language. One that I've used off an on for a number of years. Indeed, my honours thesis was written using Markdown, and transformed into a PDF using the MultiMarkdown application.

The trouble with the wikis is that they became a dumping ground for copy and pasted content. I consider Markdown to be an easy to understand and easy to use markup language. I will acknowledge that I may be biased in that opinion, especially as I'm very familiar with it. The basics are still very easy to understand and use.

The wiki contained many formatting errors. Such as missing headings, links that were formatted incorrectly, and other irritating little things. A few weeks into the course there was an attempt to at least have the students alphabetise their entries. Unfortunately, by then the bad habits had already formed.

The biggest problem was that with 100's of students all contributing content a wiki quickly became an impenetrable wall of text. Any grains of good information were quickly lost in the piles of chaff.

Discussion forums: A prompt for reflection

The intention, as I perceived it, of the discussion forums was to act as a prompt for reflection and hopefully critical thinking. Each week the lead-in for the forum was a question and some more information for consideration. There was also an entreaty to reply to at least one of the other students posts.

Once again the primary problem was one of scale. With 100's of students posting in the forum the list of posts quickly became exceptionally long. As a student it was dispiriting to try to come up with something new and original. It was also led to feelings that it was very unlikely that anyone would read your individual post.

Quizzes: Checking your level of knowledge

Each week included a quiz of 10 to 15 questions. They were predominantly multiple choice or true / false questions. Each question had provision for a set number of attempts. This allowed you, as a student, to take a guess if you weren't sure. In the knowledge that you had at least one more attempt. Except, of course, the true / false questions which only allowed one attempt (for obvious reasons).

The primary issue for me with the quizzes is that I'm not convinced that multiple choice questions are effective in evaluating student knowledge. There were no essay, or even short answer questions. As such the quiz becomes a simple mechanism to see if a student could recall facts.

The question in my mind is, does the ability to recall facts show understanding? Or does it just reflect on your ability to remember things?

Lots of questions, no answers...

The end result of my reflections is that I have lots of questions, and no real answers. I believe that the root of the problem is scale. That is, a MOOC is massive and as such is designed to cater to as many students as possible. A MOOC is also free, or at the very least available at low cost. Therefore any assessment primarily needs to:

  • Be available online, after all that is one of the words in the acronym.
  • Cater to as large a number of students as possible.
  • Work with as little investment in time and money as possible.

The end result, at least in my experience, is assessment that is not very effective. I am willing to preemptively agree that a sample size of one is not in any way a sufficient sample size. It has given me food for thought. Perhaps a MOOC isn't meant to provide in-depth understanding. Is it only meant to provide an overview of a topic, as a means of wetting a students appetite for further investigation and learning. Or if I was to be extremely cynical, is the purpose of a MOO more of a marketing exercise intended as a way to try to recruit more students to the sponsoring institution?

These reflections have got me asking many questions, and I would very much like to explore them further. I'll also be thinking about them before I sign up for my next MOOC.