Q1. We can confuse learners when we:
a. fail to actually complete the sentence we started in the question.
b. inconsistent grammar in the options.
c. sometimes veer off into another idea entirely.
d. wombats.
I found this question in a blog post by Cathy Moore, written way back in the stone age (2007).
Her 6 questions not only made me chuckle but got me thinking about the immeasurable number of terrible multiple choice questions I’ve encountered over the years.
My usual process of selecting an answer goes a little something like this:
STEP ONE: cross out the one inevitable ridiculous answer
STEP TWO: cross out the one that literally has nothing to do with the topic
STEP THREE: guess between the remaining two answers
And guess what? I did exceptionally well at school ;)
Right from infants, to high school, through to higher education-multi-choice quizzes are everywhere. We’re familiar with their structure and most of us know exactly how to manipulate them.
So why did we choose to build an entire workplace learning platform around multiple choice questions?
Because they’re god damn powerful.
But only when they’re done right.
There are a few crucial elements of writing good questions, and more importantly good answers.
Writing a good question
Recently our Instructional Design quizmaster, Joel, created the Yarno content bible. The complete how-to for creating awesome content in Yarno.
He states that a good question should be:
- Clearly, concisely and honestly written
- Free of unnecessary jargon and acronyms
- Written in conversational language
- Structured, to make learners think
- NOT negatively phrased (hehe)
So the ultimate goal of a question is not to trick the learner, but to test their knowledge, and most importantly what they should do as a result of that knowledge.
Here’s a quick quiz for you:
For a learning objective about ‘the benefits of using active voice in report writing’, pick which is the ‘best’ question to ask:
Q2. Should you aim to use Passive Voice or Active Voice in your report writing?
a. Active Voice.
b. Passive Voice.
OR
Q3. Which of the following sentences would be most effective in your report writing?
a. An investigation has begun into the issue that you have raised.
b. I have begun an investigation into the issue that you have raised.
c. The issue that you raised is now under investigation.
d. You raised an issue that I have begun an investigation into.
See where they differ?
Q2 purely fact checks, and if the learner was blindfolded, has a 50/50 chance of success. Not good.
Q3, however, is a good question! It makes the learner switch their brain on, analyse each possible answer by comparing their knowledge of what active voice looks like, then deduce the correct response.
Writing good answers
Q4. When should ‘all of the above’ be used as an answer?
a. Never
b. Not at all
c. In no circumstances
d. All of the above
There are two types of answers: the correct answer, and the incorrect answers.
A good rule of thumb is to think about whether someone that’s unfamiliar with the content would be able to spot a noticeable difference between the correct and incorrect answers.
Writing gimme answers that are obviously an incorrect choice (like 'd.wombats’ in Cathy’s example above) dramatically reduces the room to learn as learners can follow a pattern like my (terrible) method of process of elimination and guessing.
So, good answers have two effects:
- Only learners who actually know the correct answer select it
- Learners who don’t have correct knowledge are surprised when they answer incorrectly, which makes them more open to the correct knowledge in the explanation.
True or False / Yes or No answers
These kinds of answers suck. There are no two ways about it.They undermine the power of multi-choice because learners are able to reason or guess the correct answer with a great deal of confidence. The best bet is to reframe the question to avoid having to use these kinds of answers.
Writing incorrect answers
Writing incorrect answers can be difficult and takes practice. Here are two principles that may help:
- Make them plausible
Back to the wombat answer here, the worst part about it was that in no way was the answer plausible. Using plausible incorrect answers are great for triggering a ‘hey, wait a sec’ moment in the brain. Deliberation is critical thinking, and critical thinking is learning. So, though every question should have one certainly correct answer, a slight degree of‘maybe’ can help engage a learner’s brain. - Make the structures similar
Learners who don’t know the correct answer often try to figure out the correct answer based on how the question and answers are structured and worded. The best practice to avoid this is to write all the answer choices for a question to be of similar length and grammatical structure.
On question structure, there are several that can (and should) be utilised. One structure to only use with a specific learning goal in mind is ‘fill in the blank’. Despite being a valid question type, using a ‘fill in the blank’ structure does come with the riskof the learner focusing on holding the partial sentence in working memory and then trying to complete it with each alternative.
Where does the learning actually happen?
When a learner answers a question incorrectly, they’re more likely to be receptive to receiving an explanation of what the correct answer was, and why their answer wasn’t correct.
Explanations are where the real learning happens, so it’s an important investment of time to write good ones. A good explanation clearly and succinctly states the correct answer to its question, provides some context for understanding of the question, but NOTHING more!
It’s crucial that you keep it as short and sharp as possible to avoid losing your learners' attention for that explanation and damaging their respect for the process.
If you feel like you need to write a longer explanation to tick off all the information that needs addressing, consider breaking the explanation down into another question.
Good multi-choice questions deliver results
In essence, good multi-choice questions are both versatile and reliable.
Because of the format allowing complete freedom in writing questions and answers, multi-choice can assess anything from basic recall and identifying knowledge gaps, to application, reinforcement, analysis and evaluation.
When written around solid, behaviour based learning outcomes, multi-choice questions are an objective way to consistently measure the learning goals.
When learners are delivered multiple choice questions, they’re on an even playing field. One where knowledge gaps can be clearly identified, and most importantly – closed.
And that’s where we come in.
If you're ready to improve employee performance through the power of the humble quiz, give us a buzz !
