Strictly deliberate practice

Okay, I admit it, I’m an avid Strictly Come Dancing watcher and my Saturday evenings are just not the same at the minute…


One of the many things I enjoy about the show is the progress made by the celebrity dancers over the course of the programme; in many cases, it’s phenomenal. What I find even more amazing is the work of the professionals aka the experts.

The professionals have been selected due to their expertise in a repertoire of dance disciplines. Each week, not only are they required to train a novice (celebrity) how to dance a particular routine, but they are also involved in at least one other routine during the show, which they perform with gusto and grace. In order to reach this standard, they have acquired thousands of hours of practice over a number of years and, as a result, have highly organised and fine tuned schema in their long term memories, which allows them to access new routines with efficiency and ease (I’ve discussed cognitive architecture in previous posts, so don’t intend to dwell on it here).

The work of Ericson and colleagues found that for individuals to become experts in their respective domains, it generally takes about 10 years of deliberate practice.

‘Simon and Chase (1973) observed that nobody had attained the level of an international chess master (grandmaster) “with less than about a decade’s intense preparation with the game”.’

They found similar results when reviewing other domains (teaching not included of course) and concluded that:

‘the differences between expert performers and normal adults reflect a life-long period of deliberate effort to improve performance in a specific domain.’

What does an expert teacher look like?

Whilst it is acknowledged that there is currently little consensus as to what constitutes an expert teacher, one could argue that as it currently stands, it is one that gains the highest value added achievement that is likely to be considered an expert. The often discredited work of Hattie and Marzano, along with studies in the domain of cognitive science provide us with many examples of methods and approaches that have greater impact on achievement. For example, we know that feedback which looks forward and is task-centred is more effective than no feedback or ego-centred feedback. We know that testing learners on material supports their ability to retain and retrieve knowledge. We know that spacing practice supports retention better than massed practice.

Using these (and the many other research informed approaches) as a barometer for expertise is arguably a starting point for all teachers in their deliberate practice towards expertise.

What is the difference between deliberate practice and normal practice then?

Ericsson et al inform us that ‘deliberate practice includes activities that have been specifically designed to improve the current level of performance’. They go on to state that key conditions for deliberate practice include: intrinsic motivation, feedback, and focused practice on specific areas of weakness. Unlike deliberate practice, normal practice is generally unstructured and feedback-free.

As a teacher, what can I do with this information?

This blog is timely in light of the recent Deans for Impact – Practice with Purpose release for teacher educators, and using the framework suggested will provide trainees with a good foundation for deliberate practice. For those already teaching, there are many aspects of this report that you might use, but as Ericsson et al point out, practice is not inherently motivating, requires time and is effortful. Therefore, it might be something you discuss with colleagues in order to cooperatively agree targets, structure practice opportunities, and monitor and provide feedback to one another to make progress towards ‘expertise’, or rather,  a 10 from Len!

Own your room

Not all teachers have the luxury of their own classroom; many have to move from room to room for their lessons, but regardless, one of the biggest pieces of advice that I give to teachers when managing classroom behaviour is to OWN YOUR ROOM.

When I started out teaching, I would often arrive at my classroom 2 minutes before the lesson to find students already in the room, sometimes eating/drinking, on phones, generally treating the place as a common room, rather than place of learning. This put me on the back foot as a teacher. I couldn’t arrange the tables as I saw fit, so would try to involve the learners in moving the room around (mayhem). Then trying to get them to sit where I needed them became even more of an issue. I had to start negatively by enforcing rules that learners should have been following; “put your drink in your bag”, “put your phones away”, so getting learners focussed on the lesson became difficult. Basically, I was taking part in unnecessary battles, when I should have been inspiring learners to learn about my subject. So after a terrible first year, here’s what I started to do – I owned my room. Below I have put some simple strategies that can help you to do the same:

  1. Where possible, arrive at your room before your learners and if they are in the room before you for whatever reason, ask them to leave whilst you set up. Do not work around them in your classroom – even if it means delaying the lesson start by a few minutes until you are ready.
  2. Where possible, set the room layout differently to last time (or try to vary at least a little with a different seating plan). Learners get comfortable very quickly and as soon as they take control of a seat, it’s very difficult for a teacher to gain your classroom control back. In addition to this, research by Smith (1985) has demonstrated the benefits of multiple learning environments on memory. Whilst not a completely different environment, the variation in position in the room, may result in less environmental cues used for memory.
  3. Welcome every student at the door. This not only sets a positive tone for the session, but it also allows you to prevent any misdemeanours prior to them entering your classroom. At this point, you can also start to direct them to where you want them to sit. “Morning Kye, please sit there” (Note: I have not asked Kye if he would mind sitting there, but have told him politely).
  4. In most instances, I’d suggest that you begin the class swiftly with an overview of the expectations for the session. That way, there will be no surprises along the way. “Here is what we are doing today and this is what I expect from you”. Further to this, according to Marzano (2003, cited in Petty), the use of ‘reminders’ has a 0.64 effect-size on achievement and is a useful strategy for developing student-teacher relationships in the classroom. This sense of clarity with expectations for learning is supported further by the work of Wiliam on formative assessment. Thus starting most sessions in this way is desirable.
  5. Recap prior learning so that students can draw upon what they already know about the topic. Supported by a wealth of cognitive psychology research, low stakes testing offers a multitude of benefits. Not only does it allow for initial assessment to take place (if done properly*), but it also allows for learners to take part in retrieval practice. This is a low-cost, high impact strategy to support learner acquisition of knowledge, which can be built upon as the lesson progresses. In terms of behaviour, this will provide a routine for learners and even the most challenging like routine.
  6. Try to avoid large group work. When it comes to group work, then anything more than groups of 3 and I start to worry about the benefit to all involved. Slavin, Hurley and Chamberlain offer two key features of effective group work (working towards the same goal and having accountability for one another’s learning), but even so, it becomes very difficult for a teacher to manage large groups. I tend to stick to paired activities in the main, but that’s my preference. If you can be confident that all members are participating fully and are getting the most from their experience in the group (and I’m not talking ‘soft skill’ nonsense), then fine, but larger group size does create the conditions for behaviour to go awry. My ‘go-to’ strategy is think, pair, share. A great post on the strategy by HeadGuruTeacher can be found here and in using it well, the teacher maintains their control, thus their ownership of the room.


There are many more ways of owning your classroom, but I generally offer the above 6 tips for my trainees to enable them to then make decisions based upon their own contexts. I haven’t discussed classroom rules, rewards or punishments, because there’s a whole blog post in that, but these are just simple strategies that can be adopted with relative ease. If you find learner behaviour a struggle, then perhaps try owning your classroom.


* For effective initial assessment, consider using multiple choice questions, along with a whole group answer approach, whereby mini-white boards, individual hand held devices, or simply fingers up, is used to determine each learner’s starting point. Do not resort to the ‘asking an open question and only the most confident shout out’ approach.

Songs of Memory

A few weeks ago whilst teaching about Ebbinghaus’ forgetting curve and distributed practice*, one of my trainees was able to conceptualise it by way of learning songs… Here we go, I thought…


“You know when you hear a song for the first time, you can only remember a little of it, but the more you listen to it, the more you remember of the song?”


Well, yes. I suppose there may be a point to this. I thought about some examples that I could use to explain the forgetting curve and distributed practice via the ‘learning a song’ approach and here are my thoughts:


Twenty years ago, Puff Daddy (AKA P Diddy, AKA Sean Coombs, AKA whatever the latest is) and Faith Evans released ‘Missing you’ a song in memory of the Notorious BIG – Notorious! My summer was spent with the CD on loop, playing it over and over again. A 5 minute song turned into 5 weeks, until I became sick of it. To this day, I can still spit bars like the 13 year old me – word.


But was this distributed practice I thought? I mean, my whole summer was blocked with that song – I had overlearned it. Whilst I could use this to discuss the forgetting curve, I suppose frequent visiting over a long period wasn’t the best example to use for distributed practice…


I then thought about a song that I hadn’t heard as much, but with sufficient space between listening… aha! Christmas songs!


A yearly dose of Maria, George, The Pogues et al and… wait, I can’t say that I know all of the words to any of those songs… there’s bits I mumble my way through in an attempt to appear like I know, but I really don’t. Is it because I almost completely forget with such a long period between listening? Maybe.


So what could I use as an example? Well, I have struggled with this one. Would I have remembered ‘missing you’ if I hadn’t listened to it intensely for such a long time and overlearned it? Maybe I could have been more efficient with my time and had I thought about improving my memory of the song, would have listened to it a couple of times every week? Then I might have remembered a few other songs from that summer.


I can’t think of an example to demonstrate distributed practice, but I’m going to conduct a little experiment on myself and listen to the pogues twice a week up to Christmas in an attempt to get the words… then I might have a decent example to use in future. I’ll keep you posted! 

*For those of you that have no idea what I have just been on about, a summary of the forgetting curve and distributed practice can be read by clicking the links.

Do we need to know about learning theory or not?

Yes… well kind of…

Learning theory provides us with a why for the what. When we teach learners, we expect them to know the why for the what, so why wouldn’t we expect the same of teachers?

Link to image source
The problem that many practitioners face is that they probably haven’t studied learning theory since their teacher training, and I’m certain that in some cases, much of that was ‘guff’ (Gardners MI, learning styles, learning pyramids etc etc). Much of the theory that I was taught has been debunked and I don’t even know if what I was doing as a result of learning the theory was particularly useful or not. Aside from this, there are so many theories (see above image) that it becomes difficult to determine what is useful to know and what isn’t.


For this reason, I recommend that the minimum that all teachers should be armed with is some objective evidence of how students learn best. I have previously written about the different types of evidence you might use to determine how students learn (here). If we know that the reason we are doing something in our classroom is because there is some objective evidence to show that it works, then we should have confidence that we are likely doing the best for our learners. If on the other hand, there is no objective evidence, or evidence that refutes something we are doing, then perhaps we need to consider what we can do better for our learners (I’m not saying that the more subjective evidence is wrong, but there might be something we can do that will be more effective). So where can we get the more objective stuff?


Below you will find a range of the more objective evidence that I use. I have summarised each and given a subjective rating out of 5. My rating is based on the security of evidence and how useful it is for a classroom practitioner.


Article/ Research Summary My Rating
Dunlosky et al (2013): What works, what doesn’t


This document ranks some of the more effective study strategies from cognitive and educational psychology, specifically with HE learners. It’s a very accessible and a go-to document. *****
Rosenshine (2012): Principles of Instruction


An overview of 10 key principles of instruction, informed by research on master teachers and cognitive science. Gives the reader classroom application and the research base. ****
Deans for Impact (2015): The Science of Learning


Based on the research of cognitive scientists, this is another go-to article for my trainees. The content is clear and accessible with great classroom application. Informed by *****
The Learning Scientists (2016):

Six Strategies for Effective Learning


Six of the most effective learning strategies from cognitive science are simplified and visualised. Highly accessible and available in a variety of formats for teachers and learners. *****
Hattie (2012):

Visible Learning


This isn’t a link to the book, but a pretty good review. There is a wealth of research informed strategies covered in the book. Despite a lot of criticism for his methods, this is a good starting point for understanding what is typically more effective in the classroom. ***
Marzano et al (2001): Classroom Instruction that Works


This is a link to the complete book. The work summarises a wealth of classroom based research. 9 research-based strategies are explored with a thorough coverage of the research and a range of applications for the classroom. A big read, but great book. ***
Education Endowment Foundation Toolkit


This toolkit for practitioners summarises classroom based research into 3 key elements – cost, research security and months impact on achievement. Some of the research methods used by the foundation have been scrutinised, but certainly worth exploring as a starting point. **
Coalition for Psychology in Schools and Education (2015):

Top 20 Principles from Psychology for PreK-12 Teaching and Learning

This offers twenty key principles for teaching and learning based on psychology. It is accessible, but a huge document. Each principle is explained with relevance to teaching. *****


A ’21st Century FE College’

This blog post is in response to a TES FE article that has been promoted widely this week – How FE can transform the workforce of tomorrow


On the face of it, the points made in the post might sound like grand ideas, but with all due respect, I disagree with some of the points made and am concerned that there is no evidence to support an argument being made by a senior leader in the sector. For this reason, I want to break the article down to highlight the problems.
Firstly, there are many assumptions and no evidence to back up some of the statements. For example:

‘It falls to the FE sector to provide the core of the nations workforce of the future. And I would hazard a guess that these workers will need to be natural collaborators, problem solvers and out-of-the-box thinkers.’


‘In the future, chefs will probably need to work hand-in-hand with software developers to create apps to promote their restaurant…’

Making radical changes to a college based on assumptions is dangerous.


The article includes a series of sub-headings followed by a rationale for each. Let’s examine each one:


The author wants to ‘move away from classrooms that have not progressed since the days of chalk and talk – with a desk at the front, where teaching remains in the hands of teachers and students are passive recipients of information, rather than active learners.’

Seriously? Teaching in the hands of teachers is a bad thing? Let’s not have experts in the room then. Let’s just hire someone with no experience, knowledge or skills in the subject and pass the control to learners to find out for themselves. In fact, why even have a college building at all? The above comment undermines the fantastic knowledge, skills and experiences of the FE workforce and would put learners at a complete disadvantage (cue evidence to support sweeping statement): Kirschner, Sweller and Clark (2006) inform us that:

‘based on our current knowledge of human cognitive architecture, minimally guided instruction is likely to be ineffective. The past half-century of empirical research on this issue has provided overwhelming and unambiguous evidence that minimal guidance during instruction is significantly less effective and efficient than guidance specifically designed to support the cognitive processing necessary for learning.’

There’s no disputing that learners should engage in ‘active learning’, but I do wonder whether my understanding of active learning differs to what the authors is, particularly if they’re suggesting the complete removal of guided instruction. A wealth of well researched learning strategies can be found here to support learners in becoming ‘active learners’, fortunately for us teachers, there is still a large role to play.



Apparently, ‘to get students ready to compete in the global market and thrive in the sharing economy, we need to move towards self-directed learning’. Teachers as facilitators to develop independent learners of the future. To achieve this, the author advocates ‘skills-based learning’, which I am broadly in favour of, that is, if we are talking about domain specific skills. However, it isn’t clear whether the author is only talking about these skills, as littered throughout the article is reference to generic/soft, transferable skills. Despite how lovely it would be to be able to teach a learner generic skills such as problem solving, critical thinking and creativity (out-of-the-box thinking), I’m yet to see a framework for the effective teaching of these ‘generic’ skills. Whilst there’s no disputing how important these skills are, current research shows that these skills are tied to a domain and that it is very difficult to explicitly teach them. A recent blog from Carl Hendrick articulates this far better than I ever could.


As a side note, why all of a sudden are these skills more important than they’ve been previously? We are quite good at being creative and problem solving… you only have to look at advancements in the last 100 years. Have we all of a sudden lost creativity, problem solving and communication skills?


I can’t argue with this. Collaboration is a key characteristic that we should find in any learning environment. Slavin, Hurley and Chamberlain (2003) have demonstrated that cooperative learning has the potential to be highly effective, particularly if two conditions are met – firstly that learners work as a team, not just in a team and secondly, all individuals should be accountable for the learning.


Project based learning is suggested by the author as a pedagogical approach to bridge different domains. Sure. But not at first and not for all learning activities. Perhaps once students have acquired sufficient knowledge to participate as a group and be accountable for one another’s learning. Perhaps when they are on their way to mastering a topic. Research has demonstrated that experts benefit from less guidance (expertise reversal effect), whereby guided methods of instruction become less effective as learners become more experienced and knowledgeable. However, those that begin a vocational course at 16 have arguably got little experience/knowledge/skills in that domain and are therefore novices, thus a guided approach to instruction is a necessity.

With the above in mind, if we adopt ‘facilitator’ roles in a vocational education and training with novice learners, are we at risk of widening the skills gap, not narrowing it?



I’m a technology advocate, I sure am, but rather than adopting an alternative paradigm, why don’t we align it to current paradigm? (Blog to follow). The whole changing spaces idea can still work with current ‘effective’ instructional approaches, so if we must, let’s go for it.


Whilst I agree that colleges need to evolve, fundamentally, the instruction isn’t the problem, it is doing things without a sufficient evidence base that is. Influential leaders in the sector need to be careful what they wish for.

Where do you get your information from about how students learn?


This post was originally written for the TES (edited and featured 0n 25/06/16)

For most, this question may seem an odd one, but according to Dr Gary Jones, sound evidence based practice requires teachers to discriminate, apply and evaluate a variety of sources of evidence in order to answer such questions, so have you?

@OliverCavigliol and @AceThatTest recently collaborated on Twitter to determine a continuum of subjective to objective sources of information that may be used by teachers to find out about how students learn. This article intends to explore the benefits and limitations to each of these sources:


Intuition and Experience

Highly subjective, people’s intuition often derives from a desire to find patterns and connections in randomness. Sometimes people overestimate their intuition and prevent themselves from making sound decisions. Take the roulette player that observes 5 reds come in one after another, he believes that there is little chance that a red can come in again, so places his bet on black. But of course, the odds are still the same regardless of how many came before. Liken this to the classroom, we use our intuition and experiences to guide us in situations that we face day-to-day. Being time-short means that teachers don’t have the opportunity to contemplate decisions, rather they act in the moment when it comes to thinking about how students learn best. Denes-Raj and Epstein (1994) propose that people process information in two different modes, one identified by terms such as rational, analytical and deliberative, and the other by terms such as experiential, automatic, intuitive and natural – The latter being a dangerous concoction of highly subjective approaches. Conversely, experience can support us contextually, particularly when less objective sources are available to us, moreover, intuition may useful to open yourself to new ideas that rational thinking may not allow.


CPD Sessions

Though some CPD sessions are informative and outside experts can act as agents for change (, there are a lot of snake oil salespeople out there. These individuals work in their best interests to promote ideas and resources, not really providing us with unbiased information about how students learn and often not informed by research. Furthermore, there remains limited evidence to support the impact of one off CPD sessions on teaching and learning ( However, if the CPD session is well informed and part of an ongoing community of practice, then this is where it is likely to have most value.


Communicating with peers

Teacher Learning Communities are held in high regard by Dylan Wiliam, who advocates this over the traditional sheep-dip approach to CPD, but done alone, does it really tell us about how students learn? Probably not, but in trialling strategies that are informed by more objective sources, it is certainly worth working with peers in communities of practice to determine how students learn best in your context.


Media and Blogs

This very article poses some bias towards particular sources. The very nature of this publication provides all in education a voice – some more authoritative than others. Blogs can also be produced by anyone and perhaps reinforce bad practice. Having said that, with both media and blogs, the information is current and highly accessible, so as a starting point, why not use it to find out how students learn?


Popular education books

There are thousands of books on the shelf that serve to provide us with information on how students learn. The issue we face is deciphering which are the most valid and reliable sources. Of course, once you get past this gargantuan task, you then have the bias of the author and editor to tackle. In spite of this, the thing that makes books more desirable is their ability to make the research accessible.


Education journals

On the topic of research, there are an array of individual peer-reviewed studies available covering a broad range of age groups, subjects and countries. These have been synthesised by both John Hattie and Robert Marzano, who have drawn upon the thousands of studies to determine how students best learn, producing an ‘effect-size’ for each of the strategies. There are some that have critiqued the methods used to determine the ‘effect-size’, however, questioning the validity of such an approach. Despite this, it is difficult to completely dismiss the findings of such large scale studies – just don’t take it as a standalone piece of evidence.


Cognitive science journals

Out of all of the abovementioned, this resource is the only one that focusses on the brain – that is what we’re here for isn’t it? These studies try to isolate the variables associated with typical classroom experiments and are generally laboratory based, so are pretty much as objective as we can get (of course, there is neuroscience, but this is a developing, yet murky area). Key principles of the learning science can easily be applied to the classroom, but it can be problematic trying to interpret these.



In determining how students learn best, we should try to use as many of the aforementioned sources as possible, preferably using more of the objective sources. If we can draw upon, and amalgamate the information gleaned from each to determine the most effective strategies to support our learners, then surely that’s what we should be doing?


Here’s an example:

Cognitive science informs us that distributed practice is a highly effective way to increase long term retention ( Classroom experiments also corroborate this, with the effect size of spaced practice being 0.46 ( My experience tells me that cramming delivery into short blocks does little to help my learners remember the content at the end of the year and my peers would agree. So having gathered this information, next year I shall now try distributing practice and evaluate the findings at the end of the academic year.


So, how will you use the evidence to find out how students learn?

You don’t need to…

Over the last few weeks I’ve been working with a new group of unqualified, practicing teachers. It’s fascinating to hear about some of the things they’ve been indoctrinated with from others – those that did their teacher training many years ago.



Over a series of sessions, I’ve dispelled several myths about things they think they need to do and asked them to question their current practice. It’s not that this practice is necessarily wrong or ineffective, but the view of a ‘model lesson’ is, in my opinion. I’m going to explore some of these myths in this post and hopefully reassure readers that you don’t need to do any of them:


1. You don’t need to… start all lessons with a ‘starter activity’.

While it might be beneficial to grab the attention of the learners, a lesson needn’t start with an activity that has little relevance to the content. If you’re going to use one, I’d suggest a quick recap quiz for retrieval practice and initial assessment. Having said this, sometimes you might just fire straight in with the main body of the lesson and that’s fine, there isn’t a ‘right’ way to do this.


2. You don’t need to… write your learning objectives on the board.

It’s so frustrating that people think this makes a difference to the learners. Often the language used on the board is written in learner unfriendly, educational jargon. In most cases it is important to share the intentions with learners, so that they know what they’re doing and why, but sometimes you might reveal the intention as the lesson progresses. Whether you write down, tell learners or mime it, it doesn’t matter. Having said this, I often write intentions on the board so that learners have a point of reference should they wish to clarify what they’re aiming for, but I normally write this in the form of a question.


3. You don’t need to… make your learning activities fun, engaging and relevant to learner interests.

Two of my favourite quotes: ‘learning happens when people have to think hard’ (Coe, 2014) and ‘memory is the residue of thought’ (Willingham, 2009) should be considered here. All learning activities should give the learner the opportunity to think about the content. If fun, engagement and interest is a byproduct, then fine, but we should ensure that the focus is on content first. It doesn’t really matter how you do it, though some methods have, through research, demonstrated to be more effective than others (see here and here).


4. You don’t need to… worry about having enough time to teach the qualification.

This is something I hear a lot of, particularly in recent years where the guided learning hours of qualifications are being stripped back and every minute of a teacher’s contract is accounted for. For a start, you’re probably spending too much time on writing lesson objectives, doing starters and fun activities where the learners aren’t actually learning. If 5 mins are wasted in each lesson doing this and the learner has 12 lessons per week for 36 weeks, my maths says that’s 36 lost hours that I’ve just found you. In reality though, we are time short, so let’s not waste the precious time we have on nonsensical, ineffective tick box exercises.


5. You don’t need to… develop learners’ English, maths, soft skills etc in every lesson.

Whilst I am a huge advocate of developing literacy and numeracy through subject lessons, I don’t believe this should be at the expense of the content. I also don’t think we should force something in to ‘tick a box’. Natural opportunities should be taken and opportunities to develop the skills around the subject should be considered where appropriate. For example, if an learner uses subject specific terminology incorrectly, I would look to explore their understanding of the term and help them to put the word into context through use of a glossary.


6. You don’t need to… have a lesson plan.

Of course, you’d be foolish to think that you can teach without some sort of a plan, but you certainly don’t need to complete a particular lesson plan template. I’ve seen people plan to the exact minute in their lessons, but if learners don’t get something, rather than moving on because it is 9:23 and your plan says that you should be giving learners an activity, stop, and respond to what the learners need. Having a broad aim, an idea of how you’ll achieve it and how you’ll monitor learner progress towards it will allow for a more responsive approach to the learners – you might even be able to squeeze all of this information onto your fag packet.


7. You don’t need to… do what’s always been done.

New teachers, old teachers, teachers with no label – there’s an obsession with doing things how it has always been done. You pick up a new unit to teach, so follow the scheme that was planned by the teacher who did it in 2007, because that’s how its always been done. You include a learning styles inventory within your induction period and write the results on the group profile with no intention of using them, because, that’s how its always been done. Hey you that is nodding your head to this! Take control of this situation and your professionalism.


There’s probably many more myths that I dispel in every session, but these are a few that I have and will continue to challenge. You don’t need to do any of the above and don’t let anyone tell you otherwise, but equally, if you want to do them, then that’s your call.

Should we spend more time designing multiple choice questions? a) Yes b) No

I am the first to admit that when I plan my lesson resources, I spend far too long making them aesthetically pleasing. Of course, I try to ensure that my instructional design is efficient and the content is challenging, but I enjoy making the resources look great too. There are probably many others, just like me.


In this post I’d like to look at formative assessment design, specifically multiple choice questions (MCQs), and I will argue why I need to spend more time focussing on designing these and probably less time on how ‘funky’ my resources look.


Let me explain the benefits* of multiple choice questions before I go on to how you might approach the design of them:

  1. The ‘testing effect’ – Frequent quizzing has shown to enhance later exam performance (McDermott et al, 2014) as learners are provided with the opportunity of retrieval practice. There is, however, research that indicates that MCQs might not be as effective for retrieval compared to short answer response questions due to the answer being available, thus learners are not required to think as hard (Kang, McDermott and Roediger, 2007).
  2. Identifying gaps in knowledge – We can quite reliably use multiple choice tests to identify the gaps learners have in their knowledge. We can use this information to close the gaps in knowledge for groups or individuals. Whilst there is the argument for guessing answers, if we increase the number of plausible incorrect answers and the number of questions to respond to, we do increase reliability. This is illustrated in the work of Burton et al (1991):
  3. 00001Furthermore, when we ask questions to learners in class, each learner is usually asked something completely different and therefore, this results in completely different answers. Whilst I think questioning is useful, it isn’t a reliable measure of learner understanding, plus we only know the response of that one learner we ask, not the others.
  4. They can be used as a diagnostic tool – If the questions are written as such that we can determine why the learner is selecting a particular answer, then we can start to diagnose problems with their cognitive processing (Wylie and Wiliam, 2006). Wiliam advocates using a small number of these questions at a hinge point in the session, whereas I’d argue that they’re useful at any point. The problem is tha they are quite challenging and time-consuming to create, as Harry Fletcher-Wood and others have found. I’m still developing my thinking on these, but here is a video by Wiliam which explains it in a little more detail – “Kids cannot get it right for the wrong reason”.
  5. Quick, visible responses  – When the learners answer these questions, we need to be able to see the responses of all individuals. This can be done quickly and with ease through the use of mini whiteboards or holding fingers up. The teacher can view the whole class in a few seconds and determine whether they can move on, or revisit information accordingly. The wealth of formative assessment online tools also allow for MCQs to be administered to all, and automated marking can generate clear analytics quickly. I personally like Google Forms, but there are many other online tools offering similar features.


Designing the Stem

In designing effective MCQ’s I have found several research articles and documents (1,2,3,4). Each of which offer similar advice when writing questions and answers. Here are some of the key things to consider when writing the stem:

  1. The stem should be meaningful by itself and should include the main idea.

Basically, this means that the main thing you’re trying to find out about should be in the stem. E.g. What chamber does deoxygenated blood enter in the heart? (I am trying to find out if they know about chambers of the heart).


  1. The stem should not contain irrelevant material.

This will just serve to confuse learners and this can cause more harm than good. Learners may create ‘false knowledge’ if the information is not relevant.


  1. Avoid a negatively written stem.

Where negatives are used in the stem, this can make the question easier according to Harasym, Price, Brant, Violato, and Lorscheider (1992). Furthermore, negatives can cause ambiguity in what is being asked and just because a learner knows an incorrect answer, this doesn’t mean that they know the correct one. Here’s an example by Burton et al (1991).

Which of the following is not true of George Washington?

  1. He served only two terms as president.
  2. He was an experienced military officer before the Revolutionary War.
  3. He was born in 1732.
  4. He was one of the signers of the Declaration of Independence?


Designing the Answers

In a meta analysis of MCQ research, Rodriguez (2005) informs us that it is not the number of distractors but the quality of distractors that are important when designing answers for MCQs. It was found that writing more than 2 distractors can become challenging and is not significantly more effective that having more, so having A-C is fine if you are struggling to produce A-D. The key elements that should be addressed when writing stem answers are:

  1. All alternatives should be plausible.

Essentially, each incorrect answer should be plausible. In the example below, there is clearly one implausible response:

In what year was Winston Churchill first chosen as Prime Minister?

  1. 1700
  2. 1940
  3. 1941
  4. 1942


  1. Alternatives should be stated clearly and concisely.

Try to avoid unnecessary ‘waffle’, so that in interpreting the question, the cognitive burden is reduced.


  1. Alternatives should be mutually exclusive.

There should not be more than one answer that can be defended as a correct response by using correct reasoning. An example by Burton et al (1991) shows two possible correct answers:

How long does an annual plant generally live?

*a. It dies after the first year.

  1. It lives for many years.
  2. It lives for more than one year.

*d. It needs to be replanted each year.


  1. Alternatives should be free from clues about which response is correct.

Avoid including a word from the stem in the answers. This can provide a clue to the answer, and for some, they may think of it as a trick question, thus go with an alternative answer. For example:

What muscle is the agonist on a bicep curl?

  1. Bicep
  2. Deltoid
  3. Hamstring
  4. Tricep


  1. The alternatives “all of the above” and “none of the above” should not be used.

Speaks for itself as more often than not, this option is the correct answer in MCQs.


  1. The alternatives should be presented in a logical order.

The best approach suggested is numerical or alphabetical to avoid any clues as to which is the correct response. When working with City and Guilds on a project a few years ago, I was also advised to avoid having the starting letters of each answer show an obviously different response. For example, in the first set of answers, Hungary clearly stands out and this might lead learners to respond with that, whereas in the second set of responses, each starting letter is different and leaves no clues:

  1. Germany                 a. Germany
  2. Ghana                      b. Hungary
  3. Greenland               c. Poland
  4. Hungary                   d. Russia


As can be seen, writing MCQs isn’t something you can throw together 5 minutes before a lesson. To make them effective, it requires time and a number of elements need to be addressed. I’d suggest working with your colleagues to build a bank of questions.


*I have stumbled across a couple of problems with MCQs which are worth examining. @surrealyno has written a short piece on the disadvantages of MCQ’s and Roediger and Marsh (2005) also found that using MCQs could lead to ‘false knowledge’ in some students, where they believe an incorrect answer to be true. With due consideration of the abovementioned points, it certainly will mitigate against some of these concerns and I’d argue that the advantages of using MCQs outweighs the disadvantages, particularly compared to alternative methods of assessment.


A need to understand cognitive architecture.

Understanding the basics of human cognitive architecture is essential to understanding effective instructional design, but how many teachers can actually remember anything about it (that’s if they were even taught it in their teacher training)?


Whilst there is little concrete evidence for what I am sharing, after decades of psychological studies on memory, there is a general consensus amongst psychologists, along with some empirical evidence about how memory works. Much of this stemmed from the work of Atkinson and Shriffin (1968), followed by Baddeley and Hitch (1974), which I have attempted to visualise below (I did this for my students, the document can be accessed here should you wish to use):



Produced by: Dan Williams @FurtherEdagogy with reference to Kirschner et al (2006)


With instructional design we need to understand how we can use the aforementioned information to plan and deliver information to learners that will maximise their learning. I have blogged about some instructional design approaches previously, here, here and here, though for me, the Learning Scientists and Oliver Caviglioli have really nailed it with their six study strategies.



Sign learners up for the graft

I despise the first weekend of January. Not only do I get the Christmas blues, but it reminds me of my days working in the fitness industry where there would be an influx of people fulfilling their new year’s resolution to get fit. Little did they know that by the first weekend in February, the vast majority of them would no longer be members of the gym, nor did they truly understand that in order to see the results they expected, it would take months of hard work.


We live in a consumer world and January to fitness professionals is what September is like for teachers. Learners arrive at our classrooms expecting results, but not always willing to exert the prolonged effort that is needed.


Whilst writing a forthcoming blog post on observation, I stumbled across this statement in Coe’s excellent ‘Improving Education‘ publication which really ‘hit home’:

Some research evidence, along with more anecdotal experience, suggests that students may not necessarily have real learning at the top of their agenda. For example, Nuthall (2005) reports a study in which most students “were thinking about how to get finished quickly or how to get the answer with the least possible effort”. If given the choice between copying out a set of correct answers, with no effort, but no understanding of how to get them, and having to think hard to derive their own answers, check them, correct them and try to develop their own understanding of the underlying logic behind them, how many students would freely choose the latter? And yet, by choosing the former, they are effectively saying, ‘I am not interested in learning.’

Coe goes on to inform us that ‘learning happens when people have to think hard’.


Much like the gym, in FE, there will be a number of learners that leave early on, but how can we prevent this and ensure that learners are both thinking hard, and putting effort into their learning?


Here’s some ideas for you to think about using with learners at the start of the academic year:

1. Establish routines: For those working in an FE college, most learners are joining your class with no idea as to what to expect. they will be in new surroundings, with new people and this is a great opportunity to establish high expectations and routine in the classroom. Start as you mean to go on. If you have learning activities that require little effort, or if learners are allowed to put little effort in, then guess what? Yes, that will be the routine for the year. Keep the bar high for all and regardless of the outcomes, always praise those that put effort in to their learning.


2. Find out what learners know: Initial Assessment (IA) is crucial, but I’m not talking the whole putting learners through a load of irrelevant activities where the information gleaned is never used. What I’m talking about is finding out what the learners know about your subject and use this to guide the lesson. Perhaps give them an advanced organiser to help them identify gaps in knowledge, or quiz them on the content using diagnostic questioning.


3. Organise information: The more organised the information, the better. Give concrete examples and use both verbal and visual information simultaneously during instruction (dual coding) to reduce cognitive overload. Also ensure that learners are afforded the opportunity to revisit the information on several occasions over the term (spaced practice). Furthermore, learners should be given opportunities to elaborate on their understanding and transfer their knowledge to different problems. The Learning Scientists and Oliver Caviglioli have put together 6 study strategies which I highly recommend are implemented into your planning and delivery (link).


4. Test learners regularly: Our memory trace is improved when we have to work hard to retrieve information from long term memory. Therefore, we should aim to test learners frequently through mini quizzes, self testing and the like. This not only supports retrieval practice, but it also allows both the teacher and learners to identify strengths and areas for improvement. When learners start to see that their memory is improving and that they can transfer their knowledge to new situations, this can be highly motivating.


5. Show learners that you care: I’ve added this in after reading Dylan Wiliam’s ‘9 things every teacher should know’ last week. It really is important to show learners that you care about them succeeding. For some, this can be a natural thing, but for others it might not be. Positive relationships built on mutual respect and passion from the teacher can have a positive impact on learner motivation and achievement (Hattie, 2012). As Dylan says, it’s about seeing learners as people.


Good luck for the new academic year!