Should we spend more time designing multiple choice questions? a) Yes b) No

I am the first to admit that when I plan my lesson resources, I spend far too long making them aesthetically pleasing. Of course, I try to ensure that my instructional design is efficient and the content is challenging, but I enjoy making the resources look great too. There are probably many others, just like me.


In this post I’d like to look at formative assessment design, specifically multiple choice questions (MCQs), and I will argue why I need to spend more time focussing on designing these and probably less time on how ‘funky’ my resources look.


Let me explain the benefits* of multiple choice questions before I go on to how you might approach the design of them:

  1. The ‘testing effect’ – Frequent quizzing has shown to enhance later exam performance (McDermott et al, 2014) as learners are provided with the opportunity of retrieval practice. There is, however, research that indicates that MCQs might not be as effective for retrieval compared to short answer response questions due to the answer being available, thus learners are not required to think as hard (Kang, McDermott and Roediger, 2007).
  2. Identifying gaps in knowledge – We can quite reliably use multiple choice tests to identify the gaps learners have in their knowledge. We can use this information to close the gaps in knowledge for groups or individuals. Whilst there is the argument for guessing answers, if we increase the number of plausible incorrect answers and the number of questions to respond to, we do increase reliability. This is illustrated in the work of Burton et al (1991):
  3. 00001Furthermore, when we ask questions to learners in class, each learner is usually asked something completely different and therefore, this results in completely different answers. Whilst I think questioning is useful, it isn’t a reliable measure of learner understanding, plus we only know the response of that one learner we ask, not the others.
  4. They can be used as a diagnostic tool – If the questions are written as such that we can determine why the learner is selecting a particular answer, then we can start to diagnose problems with their cognitive processing (Wylie and Wiliam, 2006). Wiliam advocates using a small number of these questions at a hinge point in the session, whereas I’d argue that they’re useful at any point. The problem is tha they are quite challenging and time-consuming to create, as Harry Fletcher-Wood and others have found. I’m still developing my thinking on these, but here is a video by Wiliam which explains it in a little more detail – “Kids cannot get it right for the wrong reason”.
  5. Quick, visible responses  – When the learners answer these questions, we need to be able to see the responses of all individuals. This can be done quickly and with ease through the use of mini whiteboards or holding fingers up. The teacher can view the whole class in a few seconds and determine whether they can move on, or revisit information accordingly. The wealth of formative assessment online tools also allow for MCQs to be administered to all, and automated marking can generate clear analytics quickly. I personally like Google Forms, but there are many other online tools offering similar features.


Designing the Stem

In designing effective MCQ’s I have found several research articles and documents (1,2,3,4). Each of which offer similar advice when writing questions and answers. Here are some of the key things to consider when writing the stem:

  1. The stem should be meaningful by itself and should include the main idea.

Basically, this means that the main thing you’re trying to find out about should be in the stem. E.g. What chamber does deoxygenated blood enter in the heart? (I am trying to find out if they know about chambers of the heart).


  1. The stem should not contain irrelevant material.

This will just serve to confuse learners and this can cause more harm than good. Learners may create ‘false knowledge’ if the information is not relevant.


  1. Avoid a negatively written stem.

Where negatives are used in the stem, this can make the question easier according to Harasym, Price, Brant, Violato, and Lorscheider (1992). Furthermore, negatives can cause ambiguity in what is being asked and just because a learner knows an incorrect answer, this doesn’t mean that they know the correct one. Here’s an example by Burton et al (1991).

Which of the following is not true of George Washington?

  1. He served only two terms as president.
  2. He was an experienced military officer before the Revolutionary War.
  3. He was born in 1732.
  4. He was one of the signers of the Declaration of Independence?


Designing the Answers

In a meta analysis of MCQ research, Rodriguez (2005) informs us that it is not the number of distractors but the quality of distractors that are important when designing answers for MCQs. It was found that writing more than 2 distractors can become challenging and is not significantly more effective that having more, so having A-C is fine if you are struggling to produce A-D. The key elements that should be addressed when writing stem answers are:

  1. All alternatives should be plausible.

Essentially, each incorrect answer should be plausible. In the example below, there is clearly one implausible response:

In what year was Winston Churchill first chosen as Prime Minister?

  1. 1700
  2. 1940
  3. 1941
  4. 1942


  1. Alternatives should be stated clearly and concisely.

Try to avoid unnecessary ‘waffle’, so that in interpreting the question, the cognitive burden is reduced.


  1. Alternatives should be mutually exclusive.

There should not be more than one answer that can be defended as a correct response by using correct reasoning. An example by Burton et al (1991) shows two possible correct answers:

How long does an annual plant generally live?

*a. It dies after the first year.

  1. It lives for many years.
  2. It lives for more than one year.

*d. It needs to be replanted each year.


  1. Alternatives should be free from clues about which response is correct.

Avoid including a word from the stem in the answers. This can provide a clue to the answer, and for some, they may think of it as a trick question, thus go with an alternative answer. For example:

What muscle is the agonist on a bicep curl?

  1. Bicep
  2. Deltoid
  3. Hamstring
  4. Tricep


  1. The alternatives “all of the above” and “none of the above” should not be used.

Speaks for itself as more often than not, this option is the correct answer in MCQs.


  1. The alternatives should be presented in a logical order.

The best approach suggested is numerical or alphabetical to avoid any clues as to which is the correct response. When working with City and Guilds on a project a few years ago, I was also advised to avoid having the starting letters of each answer show an obviously different response. For example, in the first set of answers, Hungary clearly stands out and this might lead learners to respond with that, whereas in the second set of responses, each starting letter is different and leaves no clues:

  1. Germany                 a. Germany
  2. Ghana                      b. Hungary
  3. Greenland               c. Poland
  4. Hungary                   d. Russia


As can be seen, writing MCQs isn’t something you can throw together 5 minutes before a lesson. To make them effective, it requires time and a number of elements need to be addressed. I’d suggest working with your colleagues to build a bank of questions.


*I have stumbled across a couple of problems with MCQs which are worth examining. @surrealyno has written a short piece on the disadvantages of MCQ’s and Roediger and Marsh (2005) also found that using MCQs could lead to ‘false knowledge’ in some students, where they believe an incorrect answer to be true. With due consideration of the abovementioned points, it certainly will mitigate against some of these concerns and I’d argue that the advantages of using MCQs outweighs the disadvantages, particularly compared to alternative methods of assessment.


A need to understand cognitive architecture.

Understanding the basics of human cognitive architecture is essential to understanding effective instructional design, but how many teachers can actually remember anything about it (that’s if they were even taught it in their teacher training)?


Whilst there is little concrete evidence for what I am sharing, after decades of psychological studies on memory, there is a general consensus amongst psychologists, along with some empirical evidence about how memory works. Much of this stemmed from the work of Atkinson and Shriffin (1968), followed by Baddeley and Hitch (1974), which I have attempted to visualise below (I did this for my students, the document can be accessed here should you wish to use):



Produced by: Dan Williams @FurtherEdagogy with reference to Kirschner et al (2006)


With instructional design we need to understand how we can use the aforementioned information to plan and deliver information to learners that will maximise their learning. I have blogged about some instructional design approaches previously, here, here and here, though for me, the Learning Scientists and Oliver Caviglioli have really nailed it with their six study strategies.



Sign learners up for the graft

I despise the first weekend of January. Not only do I get the Christmas blues, but it reminds me of my days working in the fitness industry where there would be an influx of people fulfilling their new year’s resolution to get fit. Little did they know that by the first weekend in February, the vast majority of them would no longer be members of the gym, nor did they truly understand that in order to see the results they expected, it would take months of hard work.


We live in a consumer world and January to fitness professionals is what September is like for teachers. Learners arrive at our classrooms expecting results, but not always willing to exert the prolonged effort that is needed.


Whilst writing a forthcoming blog post on observation, I stumbled across this statement in Coe’s excellent ‘Improving Education‘ publication which really ‘hit home’:

Some research evidence, along with more anecdotal experience, suggests that students may not necessarily have real learning at the top of their agenda. For example, Nuthall (2005) reports a study in which most students “were thinking about how to get finished quickly or how to get the answer with the least possible effort”. If given the choice between copying out a set of correct answers, with no effort, but no understanding of how to get them, and having to think hard to derive their own answers, check them, correct them and try to develop their own understanding of the underlying logic behind them, how many students would freely choose the latter? And yet, by choosing the former, they are effectively saying, ‘I am not interested in learning.’

Coe goes on to inform us that ‘learning happens when people have to think hard’.


Much like the gym, in FE, there will be a number of learners that leave early on, but how can we prevent this and ensure that learners are both thinking hard, and putting effort into their learning?


Here’s some ideas for you to think about using with learners at the start of the academic year:

1. Establish routines: For those working in an FE college, most learners are joining your class with no idea as to what to expect. they will be in new surroundings, with new people and this is a great opportunity to establish high expectations and routine in the classroom. Start as you mean to go on. If you have learning activities that require little effort, or if learners are allowed to put little effort in, then guess what? Yes, that will be the routine for the year. Keep the bar high for all and regardless of the outcomes, always praise those that put effort in to their learning.


2. Find out what learners know: Initial Assessment (IA) is crucial, but I’m not talking the whole putting learners through a load of irrelevant activities where the information gleaned is never used. What I’m talking about is finding out what the learners know about your subject and use this to guide the lesson. Perhaps give them an advanced organiser to help them identify gaps in knowledge, or quiz them on the content using diagnostic questioning.


3. Organise information: The more organised the information, the better. Give concrete examples and use both verbal and visual information simultaneously during instruction (dual coding) to reduce cognitive overload. Also ensure that learners are afforded the opportunity to revisit the information on several occasions over the term (spaced practice). Furthermore, learners should be given opportunities to elaborate on their understanding and transfer their knowledge to different problems. The Learning Scientists and Oliver Caviglioli have put together 6 study strategies which I highly recommend are implemented into your planning and delivery (link).


4. Test learners regularly: Our memory trace is improved when we have to work hard to retrieve information from long term memory. Therefore, we should aim to test learners frequently through mini quizzes, self testing and the like. This not only supports retrieval practice, but it also allows both the teacher and learners to identify strengths and areas for improvement. When learners start to see that their memory is improving and that they can transfer their knowledge to new situations, this can be highly motivating.


5. Show learners that you care: I’ve added this in after reading Dylan Wiliam’s ‘9 things every teacher should know’ last week. It really is important to show learners that you care about them succeeding. For some, this can be a natural thing, but for others it might not be. Positive relationships built on mutual respect and passion from the teacher can have a positive impact on learner motivation and achievement (Hattie, 2012). As Dylan says, it’s about seeing learners as people.


Good luck for the new academic year!

Less is more…

I’ve been writing a lot about cognitive architecture and instructional design of late (here, here and here). Arguably, the goal of instruction is to help students be able to explain information that they’ve retained and to transfer this to solve problems. Here I want to discuss a method of instructional design which may be highly effective in supporting learners with not only their retention of information, but also transfer.


According to Mayer et al (1996):

‘a common instructional practice is to provide a lengthy verbal explanation, such as a textbook passage or a classroom lecture…[in doing this]… instructors may believe…[that this]… fulfills their responsibility to provide information to the learner…[however]… this practice is not very efficient for many students.’

In their work, Mayer and colleagues conducted three experiments with a group of college students to explore the effectiveness of different instructional approaches to share a scientific explanations:

Experiment 1: Students read a summary that contained a sequence of short captions with simple illustrations depicting the main steps in the process of lightning. Students recalled these steps and solved transfer problems as well as or better than students who received the full text along with the summary or the full text alone.

Experiment 2: Taking away the illustrations or the captions from the summary reduced its effectiveness.

Experiment 3: Adding additional text to the summary reduced its effectiveness.

From the results of the experiments, it was concluded that multimedia learning that is concise, coherent, and coordinated, aids explanation recall and problem solving transfer. It is suggested that the reason for this is simply because summaries reduce the load on the cognitive system, enabling learners to carry out the cognitive processes necessary for meaningful learning, similar to that of dual coding.


In layman’s terms, an effective method of instruction is to provide learners with a storyboard of a process that contains both visual and text information, being mindful of the three ‘C’s:

  1. Conciseness: only using a few images and sentences in the storyboard.
  2. Coherence: Images and sentences should be presented in a cause-and-effect sequence
  3. Coordination: Images should be presented next to its corresponding sentences


So, I asked a colleague of mine, Mike Tyler to trial this approach and see how he found  and here’s the result…

Of course, it’s easy for me to sit here and say that sound instructional design should look a certain way, but there are implications in doing this. We know the opportunity of using resources like the above is good, but at what cost? I asked Mike a few questions following his work to untangle this a little.

1. How long did it take you to produce? 
Mike’s response: ‘It took a couple of hours… I used PowerPoint to make the pictures then grouped them, saved each one as a png and imported each into Word. Finally I saved it all as a pdf. Basically, it was a full-on mission, as they say! I’d do it by hand next time and scan / upload.’
2. In producing it, how did you know what to put where and what information to omit/include?
Mike’s response: ‘I included only what was minimally necessary to make sense of the process. i.e. osteobalsts, osteoclasts, etc). I had able Level 3 learners in mind.’
3. Do you think you will use this when teaching the process in future? Why?
Mike’s response: ‘I will probably give this a go in the future. I might try it for Anatomy & Physiology this coming term, maybe in a lesson on the energy systems.’
I’m going to be following up with Mike once he’s used this with his learners and will keep you posted by updating this post, but in the meantime, why not have a go yourself?
With thanks to Mike Tyler for this collaboration.

Three learners – What instruction?

I’ve written previously about the differences between expert and novice learners. You may want to read this if you haven’t already, as it provides a good base for this post. 

For this post, I’d like to start with a question. We have three learners; what instruction is likely to be most beneficial to them?

  • An A’ level physics learner with ten A-A* grades, including GCSE Physics
  • A BTEC Level 2 Sport learner with a range of GCSEs at grade D or below
  • A Level 1 Automotive adult learner with no formal qualifications, but many years experience of working on their family and friends cars

Of course, the answer isn’t quite as clear cut as one would like to think. I want to highlight that there is no ‘best way’ to teach in FE, all of these learners are very different and will require a different approach. Let’s explore each learner with some suggestions as to what might be more appropriate:


  1. It can be assumed that the A’ level Physics learner will have a sound understanding of the foundations of Physics. They are moving towards developing ‘expertise’, so will have well structured schema in this subject. So how best will they learn?


Kalyuga et al propose a phenomenon known as ‘the expertise reversal effect’ which I have attempted to depict below. In essence, when we have a solid foundation of knowledge in a subject, we need less guided instruction. The reason for this is that studies exploring the effect of guided instruction on experts have shown a negative impact, with some theorising that it is due to a greater extraneous cognitive load (basically too much non-relevant information), interfering with existing schema. So with this in mind, providing learners with less guidance and more opportunity to work independent of the teacher on problem solving and inquiry based tasks may be more effective.



  1. With the BTEC Sport learner on the other hand, we can assume from their GCSE profile that their schema is less organised compared to the A’ Level learner. So how will they best learn?


They probably have little knowledge of sport studies in general and therefore will require more guided instruction. As I mentioned in my previous post, without sufficient prior knowledge, minimally guided instruction is largely ineffective. To enhance the guided instruction, one should attempt to use approaches that reduce the burden on the working memory. This might include taking advantage of the ‘dual coding‘, by providing learners with visual and auditory information, and through ‘chunking‘ the learning coherently.


  1. The adult learner poses a more complex issue. They are likely to have some structure to their schema around automotive through their experience of working with cars. There are however, likely to be some misconceptions and potentially ‘bad habits’ as a result of this experience. How best to approach these learners then?


Through any instruction, this learner is likely to experience what Waxer and Morton call ‘cognitive conflict’, which essentially means the uncertainty we have when faced with new information that contradicts what we believe already (our current knowledge and experience). In terms of instructional methods, Bell and colleagues found positive results in using a constructivist approach to teaching called ‘diagnostic teaching’. This type of teaching involves:

‘lessons typically begin with a problem that exposes the variety of students’ existing thinking. Students are then confronted with the cognitive conflicts that emerge from these different ways of thinking. New insights are constructed through reflective discussion, leading to deeper understanding. This approach is challenging for teachers but research shows that it develops connected, long-term learning in their students.’

This blog by Nick Rose finds conflicting evidence in the research for teaching cognitive conflict, showing the benefits of both minimally guided instruction (diagnostic teaching) and guided instruction. In light of this, I would argue that the abovementioned learner will require a combination of both types of instruction. In the first instance to correct misconceptions and clarify understanding, explicit guided instruction is recommended. The learner is not an expert and therefore needs to have strong foundations built and reinforced. Following this, they can challenge cognitive conflict through a constructivist approach such as diagnostic teaching.


Key to planning instructional design is knowing your learners. This isn’t a case of their learning style or any other nonsense about how they like to learn, nor is it about trying to make the learning activities more ‘learner-centred’ because it is in-vogue. Rather it is about finding out as much as possible about what the learners already know and how secure they are with this. If we can do this, it will assist us in designing effective instruction to maximise future learning – whether this be fully, or minimally guided.

FE Bloggers

A blog is an identity, a way to express yourself, a way to think freely, to reflect and to share this with a large population. Your voice isn’t lost in the myriad of levels within an individual institution, but instead, one is considered an equal amongst colleagues from across the sector. I have learnt a vast amount through writing/reading and taking part in dialogue around my own and other’s blog posts, so am an advocate for celebrating those that currently blog and for encouraging others to start blogging in order to broaden the knowledge pool we can access. 


After receiving several requests lately asking for FE specific bloggers, I have took it upon myself to collect information about all blogs by FE practitioners (well those that replied to my request anyway). I have ordered them alphabetically for ease of reference.

Name: Blog Title: Link to blog: Summary of blog content (What is it about? Who is it aimed at? How often do you post?):
Bren Prendergast bjpren/navel gazing An eclectic mix, but generally SpLD/SEND, SEND Law & parental perspectives. Aimed at those not wishing to tick generic boxes & those wishing to understand SEND legal duties. I post erratically, but fairly frequently.
Carolyn O’Connor myfethoughts The blogs on this site are about further education. It is aimed at anyone interested in FE. I generally post a blog once a month.
Clare ClaresELTCompendium Aimed at teachers of English for Academic Purposes, Study Skills, teachers interested in CPD. I provide free worksheets & lesson materials, as well as discussion posts on key issues in EAP and teaching study skills. I blog every couple of weeks, sadly not as regularly as I would like!
Dan Williams FurtherEdagogy The blog has a teaching, learning and assessment focus, and using a wealth of research aims to question and challenge widely held beliefs in FE primarily, but also education generally. I post twice weekly. One a written piece and one an interview with an educator.
Diana Tremayne Another Fe Blog General FE musings with a particular focus on ESOL. Aimed at anyone who wants to read it! Posts approx 1 per month but varies according to time and energy.
Eddie Playfair Posting about education, culture and politics fairly regularly.
Emma Bell EMMAths Maths, teaching. Don’t post as often as I’d like, but I’m changing that!
Gemma Whitelock Mrs Arty-Textiles Teaching and education support for teachers of art and textiles. I post every term (not as often as I’d like to!) There are student friendly resources on the site and directory of artists to support project research. I often get emails asking for advice in designing and writing curriculum plans and projects for exam groups. . More than happy to help 😊
Gemma Whitelock (Lincoln College of Art) Sharing TLA A new sharing blog to create a platform for FE staff to share the good stuff, show and tell the positives and provide links to the things that have worked so that others can benefit. We post regularly throughout term time.
Geoff Petty Geoff’s Blog Teachers in FE or schools. Posts occur irregularly, when I have something to say
Hannah Tyreman Hannah Tyreman This blog has posts on my CPD work in FE but there are also posts on teaching & learning, leadership and technology. It’s aimed primarily at a wide FE audience, although posts also have relevance outside of this sector. I post when I have something to say or share.
Hilary Nunns Can Do Courses Irregular blogs aimed at teachers/support staff. I write about ADHD/SEND and behaviour in class.
Howard Scott Cantankerousman Reflections on lifelong learning, FE contexts, technologies and mobile learning, literacies, culture, meandering nomadic posts. Posts are irregular these days – once a month, more or less. The aim is more personal: to help assimilate things I learn – a reflective process, rather than try to prescribe (I hope I’m not cantankerous. It was created for my MA digital cultures course and a spur of the moment name!).
James Clay e-Learning Stuff Thoughts, views and news on using technology to support, enhance and enrich teaching, learning and assessment. Aimed at staff in FE as well as HE mainly of benefit to teachers and managers. Posts roughly once a week.
Jayne Stigger FE Culture Issues in education, mostly Further Education; it covers Teaching, Support Staff, E & D, Maths, Funding etc., but things of interest too. A mix of posts, interests and passions.
Joe Baldwin IncludEDfe Weekly posts about SEND, inclusion, additional learning support, high needs and ALS funding in FE.

I have only just relocated my blog from StaffRM to its new home so content is currently sparse but will be moved across in chunks!

Jonathan Wells Functional Skills – Post-16 maths and English 9 years old, average 1 post a week. Maths and English related news and comment for the post-16 sector – FE and Training providers.
Kay Sidebottom Adventures in Lifelong Learning Further education and lifelong related posts; aimed at educators from all sectors and trainee teachers. Specific interest in posthuman approaches to curriculum and identity/diversity issues. Post every month or so.
Lou Mycroft TeachNorthern A clarion call to dancing princesses, anti-heroic leaders and all who are interested in education for a social purpose. Also stuff about thinking environments and other pro-social approaches to critical pedagogy. Can be ranty but will always give you the feels.
Martin King inspirenshare Weekly Posts about technology, education, creativity, innovation and what I put into practice
Matt Bromley M J Bromley’s Blog Education, education, education – here I curate my articles and blogs about all aspects of education (pedagogy and leadership; primary, secondary, FE, and HE). I post sporadically but usually several times a month.
Mike Tyler The Residue of Thought A blog on education from an FE Sport Lecturer. The blog is not subject and phase specific, but also includes my thoughts on education more broadly. I post roughly once a month.
Nicky Hawkins nhawkinsblog Various topics, all FE related. Teachers, teacher educators, those involved in mentoring or coaching. Every 3 months or so.
Patrice Miller The teacher diaries Blog about teaching English to FE learners. Aimed at FE GCSE and FS English teachers. I try to blog monthly.
Paul Warren Paul Warren Video Blog dealing with mainly with learning technology in
FE but being expanded to include more subjects/areas
Peter Ford Education: the sacred and the profane A blog about education aimed at fellow education professionals. Posting fairly frequent (less so this year).
Phil Bird Phil Bird Ideas and tools to use in the classroom – it’s aimed at teachers, particularly those teaching ESOL, though much may be relevant to Functional Skills. I’m trying to post monthly.
Sam Shepherd Sam Shepherd’s Blog Posts about ESOL, adult learning, teacher training and issues in FE generally, posting generally twice a month, sometimes more.
Sarah-Jane Crowson Hcascholarship This blog is all about scholarly activity at Hereford College of Arts
Steven Keevil Teacher Learner Potentially it’s about focusing on Level 1 teaching and SEBD withing F.E. Its aimed at myself and anybody else who accidentally reads it.

Minimal guided instruction

Using problem-based, experiential, and inquiry-based teaching is likely to be ineffective with novices. 


My last post explored the difference between novices and experts, demonstrating that they think and act very differently due to a contrast in their knowledge and experience in a subject area.


In Further Education, specifically in vocational areas, learners arrive with little to no knowledge/experience in their subject. Take Engineering, Automotive, Hair and Beauty and Construction for example – likely to have never been studied previously.  Then there are subjects where there may be prior knowledge/experience but many misconceptions, for example English and maths. Therefore, learners are arguably still novices when they join us…


The thing is, there seems to be an obsession in FE to teach learners as if they are experts. CPD sessions across the country are riddled with the promotion of minimal guided instruction methods such as: discovery based, problem based, experiential and inquiry-based learning. I get it, I really do. We are trying to reach an audience that is getting hdownloadarder to reach, so if we can make the learning interesting and give learners more autonomy, then we might just crack the problem…

‘You will assume the role of a Wella colour expert and figure out what is wrong with Deirdre’s highlights’


The problem is, we are not doing them any favours by doing this. Once learners have a solid foundation and begin to develop expertise, then these approaches to learning may be very effective, as they can draw upon prior knowledge/experiences to assist them with their learning. Novices on the other hand don’t have this knowledge and experience to draw upon. In fact, it is likely that they will have misconceptions about the subject that, when applied to a problem based activity, may result in further confusion.


Future posts will examine effective methods of guided instruction, but for now I introduce you to a paper by Kirscher and colleagues explains in greater depth why minimally guided instruction is not an effective instructional design for those with limited knowledge. I have attempted to summarise this visually below:


So to end my post. Many of our learners are novices and need guided instruction. When they are experts, we can reduce the guidance we give.

*Also, I’d like to add that I’m not completely averse to this type of instruction on occasion, when I feel that learners have sufficient knowledge.

Experts and novices


This week I stumbled across a fantastic article online written by a self-taught card counter (Steve Pavlina) who, when reflecting on the Blackjack table was able to draw upon some lessons for life. I read this article and it immediately resonated from an educational perspective too.

Steve begins the article by outlining his fascination with the game and went on to outline how he became an expert at beating the casino:

‘I bought a book on blackjack, learned the rules of the game, memorized the basic strategy, and then studied a simple +/- card counting system. It took a heck of a lot of practice and was tedious to learn, but I eventually felt comfortable with it…Between Vegas trips I studied blackjack and card counting ever more deeply. I read 10-12 books on the subject and mastered different counting systems (Thorpe, Uston, Revere, etc.). I practised advanced counting systems that keep a side-count of aces. I drilled myself until I could count down a deck of cards in under 14 seconds. I learned to vary the play of hands according to the count, memorized optimal strategies for different rule sets, and learned the subtleties of the game that would increase my edge even the slightest degree. We’re talking a total edge of maybe 1%.’

Steve made some observations whilst playing. Below I have attempted to make sense of these through an education lens.

1. Novices will make correct decisions most of the time – It was observed that most of the time (80-90%), novices would make the same decisions as an expert, but cumulatively that 10-20% they make incorrect decisions have a big impact on their losses.


In education, we may assume that learners are learning well if, in most cases, they answer questions correctly, or produce a lot of work. Aside from these being generally poor proxies for learning (Coe, 2014), learners themselves may also believe that they’re doing well; mistaking their ability as superior to what it is (the Dunning-Kruger effect). This is dangerous because it’s the bits they may be getting wrong that cumulatively have a considerable impact on future learning (the 10-20%). Taking even the smallest misconception forward could make future learning less clear and more difficult.

Illustration by Oliver Caviglioli

For example, upon taking students into my Biology class, I have found many to arrive with the belief that all arteries carry oxygenated blood. Whilst in the vast majority of instances this is correct, it is a misconception that could cause confusion when later learning about pulmonary circulation, where in fact the pulmonary artery carries deoxygenated blood. The misconception should be corrected to ‘arteries carry blood away from the heart’, thus removing the confusion about oxygenated/deoxygenated blood. So what I’m getting at here is that we as teachers are supporting our learners’ development from novices to experts by not making assumptions about learning (as a result of insufficient assessment) and not allowing misconceptions to leave our classrooms.


2. Novices miss golden opportunities – It was observed that novices lost more money on the blackjack table due to a lack of understanding about when to gamble more and when to go bust; instead they tended to play it safe. Experts on the other hand would go bust more often and gamble high when the time was right. They used their knowledge of the odds to their advantage.


Daley found in her research of novice and expert learning that novices are ‘scared to death [and] terrified of making mistakes’, and that they want to be told what it is they needed to know in their learning. They are risk averse and as such don’t like to put themselves in positions where they may make a mistake. On the other hand, experts adopted a more constructivist approach to their learning, assimilating new information with old through experience, and because of a solid base of prior knowledge they were more inclined to know when to make calculated risks (or take golden opportunities). This is why it is essential that there is sufficient hand holding and teacher led instruction to ensure that the learner is provided with the key knowledge that they need, in order to develop into experts. Effective scaffolding should be slowly removed over a series of weeks/months to enable learners to become less dependent on the teacher and support their transition towards being an expert.

Illustration by Oliver Caviglioli


3. Novices don’t put in the time to fully understand the game – Novices don’t take the time to master the basics, whereas experts put in hours of practice and understand the basics and the more nuanced elements of the game.


Deliberate practice is crucial to becoming an expert according to Ericson et al who states that ‘many characteristics once believed to reflect innate talent are actually the result of intense practice’. Many novices (myself included) may be subject to the Dunning-Kruger effect so are misinformed and feel that they may not need the practice to master something. Our duty as teachers is to not only provide time to practise, but also encourage learners to understand the benefits of doing so (more on this below).

Illustration by Oliver Caviglioli



4. Experts are more disciplined – Experts tend to be more consistent in making decisions and taking action. Experts understand that you can make the correct decision and still lose, but they focus on making correct decisions, not on trying to force a particular outcome


In his book, David Didau (2015) informs us that ‘we are predisposed to examine the surface structure of a problem rather than recognising that its underlying deep structure is the same as something we already know’. In essence, when approached with a new problem, unless we are an expert, we are less likely to make links with existing knowledge and prior experiences to solve a problem. Novices simply don’t have sufficient information to draw upon and so can’t make informed decisions, thus focusing on the detail, whereas experts are more likely to focus on the structure of a problem and take a more consistent approach. For example, if given a maths problem to solve, the expert may think of similar problems they’ve faced and compare the structures to help them make sense of the information, whereas a novice may just try to tackle the problem without an idea of what they’re trying to find, or what the outcome might be. With this in mind, teachers need to be modelling explicitly how to approach problems making use of prior knowledge, before scaffolding problems for learners with support mechanisms that can be removed once experience is acquired.

Illustration by Oliver Caviglioli



5. Private victory precedes public victory – Experts spend a lot more time practising, which takes tremendous patience. Their real victories aren’t at the blackjack table, but in their homes practising.


As mentioned above, expert performances only arise through dedicated and deliberate practice. This according to Ericsson et al requires motivation and perseverance, which in itself is problematic, particularly if we want learners to engage in deliberate, directed practice outside of the classroom.

‘Deliberate practice is not inherently enjoyable and that individuals are motivated to engage in it by its instrumental value in improving performance. Hence, interested individuals need to be engaging in the activity and motivated to improve performance before they begin deliberate practice.’

So our role as educators is to establish an environment where learners focus on long term improvement through having a high self-efficacy for learning. To avoid learned helplessness and to encourage a high self-efficacy we should guide students towards success through modelling, scaffolding and giving sound feedback to help move them forward.

Illustration by Oliver Caviglioli


In summary, to support our learners from novice to expert we need to treat them as a novice initially and not as an expert. If we try to teach our novice learners to be scientists by giving them inquiry based science projects to complete, or treat them as hair stylists by placing them straight into a hair salon, they will act as novices (Kirschner et al). I believe, based upon what I have written (here, here and here) that the following approaches should be taken to support our learners to become experts:

  • We are experts in the subject matter ourselves
  • We plan the learning to maximise long term retention (distributed and interleaved practice)
  • We model correct practice and chunk the learning to reduce cognitive load
  • We scaffold difficult concepts to enable learners to more easily understand, before slowly removing the support mechanisms to allow greater independence
  • We provide regular opportunities for retrieval practice
  • We provide learners with sufficient time and space to practise, hone their skills and take necessary risks
  • We support the transfer of knowledge and skills within the subject through well planned and scaffolded activities.
  • We conduct regular checks on all learners’ understanding which goes beyond that of superficial questioning/observation
  • We provide task-oriented, rather than ego-oriented feedback in a timely and specific manner to move learning forward
  • We involve learners in their own assessment and one another’s against clear success criteria
  • We actively encourage learners to practise beyond the classroom through challenging homework that feeds into future lessons


Special thanks go to Oliver Caviglioli for his brilliant visuals to support the text.

Lesson objectives – what are we measuring?

“The lesson objectives could have been written a bit more measurable.”


My questions I want to answer in this blog post are the following:

  • What does that statement mean?
  • Do learning objectives need to be written so that they’re measurable?
  • How should we write lesson intentions to maximise learning?


Or should I set myself an objective? By the end of this post I will:

  • Be able to identify what a measurable lesson objective is.
  • Be able to analyse the impact of measurable objectives.
  • Be able to identify different methods of writing lesson intentions.


The comment in the title was made during a recent joint observation. Whilst on the surface it appeared to make sense, upon reflection, I’m not convinced by it and would like to explore it further.

Current educational ideologies (particularly in vocational education) lead to a primarily product based curricula, whereby meeting behavioural objectives forms the basis of our teaching, with teachers accountable for and judged on their ability to produce results as opposed to the more developmental, process based curricula. I don’t necessarily have a problem with this, in fact, I’m broadly in favour of this type of curricula,  but whatever curriculum approach is chosen, I do have a problem with being told that lesson objectives/outcomes/intentions/anything else you want to call them, should look a certain way. Many in the FE and Skills sector (perhaps education more generally) see learning as a linear and singular process of moving learners from A to B in a lesson and once they’ve achieved a desired outcome, then it is assumed that the learners have learnt. This view is wrong. Learning is liminal (particularly in post-16 education), with learners in a continual stage of development towards a longer term outcome over a series of lessons. I’ve said before that if ever a definition of learning could be agreed, it would certainly involve something about knowledge acquisition and probably something to do with long term memory and being able to retrieve information. Therefore, learning does not happen in isolated lessons.

 As David Didau (2015, p.279) notes:

all too often our learning intentions are lesson menus: here is what you should know or be able to do by the end of today’s lesson. Unless we have very low aspirations for our students, they are unlikely to do more than merely mimic the understanding or expertise we want them to master.’ David goes on to say thatif we were to share our intention for students to learn threshold concepts, then we could tell them that it might take them weeks to wrap their heads around such troublesome knowledge’.

This is also supported by Hussey and Smith, who argue that:

‘learning outcomes cannot be defined with the kind of precision that has been supposed, that they stand in need of interpretation within a context… The idea, currently popular—that first year degree students must describe, second year students must explain and evaluation should characterise their work in the third year—must be replaced with the idea that these activities are visited and revisited as the students’ progress and in accordance with the requirements of the subject matter.’

Indeed Hattie (2012) cites a very low (0.12) effect-size for behavioural objectives. With the aforementioned in mind, the whole notion of measurable objectives in a single lesson is beginning to look absurd. 


According to Hattie (2012) however, there are five essential components to learning intentions and success criteria to support effective learning, these are: challenge, commitment, confidence, high expectations; and conceptual understanding. I have said before that I’m not convinced about success criteria here, but in Wiliam and Thompson’s (2007) work, they hold the work of Wiggins and McTighe (2000) in high regard. This work advocates a two-stage approach to creating and sharing the learning intentions with learners. This includes clarifying the learning goals (what is worthy and requiring understanding?), and establishing success criteria (what would count as evidence of understanding?). Perhaps it is the success criteria where things become measurable?


I think about my own practice as a teacher trainer. If I were to teach say, formative assessment and set my objectives as:

‘To understand the 5 key strategies of formative assessment’ 

This is certainly worthy of understanding. Then if I were to make the success criteria measurable:

‘learners will be able to identify 5 key strategies for formative assessment’

 ‘learners will be able to explain the 5 key strategies of formative assessment’.

Whilst this is measurable and learners may be able to do both of these by the end of the lesson, this would be merely a performance and not learning, moreover, there may be learners that can critically analyse 3 strategies (beyond the success criteria) and know very little about the other 2. Does this mean that the lesson has failed? Of course not – it all seems rather short sighted and restrictive.


Fuch and Fuch inform us that:

‘teachers may prefer short-term goal measurement because it is easier to understand and it guides instruction more directly by providing information about when to progress from one skill to another [however] short-term goal measurement may be misleading: While students master a series of instructional objectives, progress on more global indices of achievement may be limited, failing to reflect this gain’.

For this reason, I ask myself whether learners would benefit more from a question or testing a hypothesis over a series of lessons:

What makes formative assessment effective? (question)

Formative assessment is only effective when feedback is provided (hypothesis)


With something like the above, the lesson intention is broad in a sense that it allows for a range of outcomes in the lesson and over a series of lessons, but tight enough to focus the learners on the content and be clear with what they’re learning about. Clarity is key. I believe that learners should know what they are doing in the lesson and why they are doing it. I mean you wouldn’t bake a cake without knowing the kind of thing you’re after and you wouldn’t go on a journey without knowing the destination, but does the way you write this on your lesson plan or white board really benefit anyone? It becomes a tick box approach – something we need to move away from in education.


Oh by the way, did we all meet the lesson objectives?


Unlinked reading:

Didau, D. (2015). What if everything you knew about education was wrong? Carmarthen, Wales: Crown House Publishing Limited.

Hattie, J. (2012). Visible Learning for Teachers: Maximizing Impact on Learning. Oxon, UK: Routledge.

The Cornell Method

When presenting new information to students in lessons, how can we best support the retention of this information? 

This post features a note taking strategy that may be new to many, the Cornell method. In essence, the strategy involves learners dividing their paper into two columns with a row across the bottom (as shown):


The learners should then take the following steps in order to take notes on the lesson:

Step 1 (right hand column): Take clear and concise notes on lesson content. Sentences should be no longer than 5-10 words long and the notes taken should not be copied verbatim from the presentation, unless a crucial point.

Step 2 (left hand column): Here is where learners are given time at key intervals or at the end of the session to create questions or cues that clarify meaning, or reveal information about the notes. For example:


Step 3 (bottom row): Summarising the notes in the bottom row helps to consolidate understanding of them. This is best done after the class. Encouraging learners to elaborate on the points made in the notes column will help to reinforce meaning, or identify gaps where further study is required.

Step 4: Learners should regularly revisit notes, covering the right hand column and testing themselves on the questions in the left hand column.


Why might this method effective?

In his book ‘Classroom instruction that works‘, Robert Marzano shares with us the features of effective classroom instruction. With an average effect-size of 1.0, note taking methods are really worth exploring further in our classrooms. According to Marzano, effective note making involves the learner summarising information being shared by deleting and substituting the information in order to create their own meaning. The Cornell method certainly allows for this, both in the notes section (step 1) and the summarising section following the lesson (step 3).
Beecher (1988) examined the research on note taking and found mixed results, with one such issue with note taking being that learners tend to copy verbatim. In doing this, learners are not engaging in the synthesis of information being presented. However, if used properly, the Cornell method overcomes this by restricting the information that learners write and requiring them to summarise well.


Despite the mixed results found, Beecher’s subsequent findings on reviewing notes makes for interesting reading:

‘The research findings on whether note-taking promotes encoding have been mixed. Hult et al. (1984), for example, found that note-taking does involve semantic encoding; but Henk and Stahl (1985) found that the process of taking notes in itself does little to enhance recall. They found, however, that reviewing notes clearly results in superior recall.

As we know from the work of Dunlosky et al (2013), self-testing is a highly effective study method. Therefore, if we combine the self-testing as the review element of note taking, we may be on to something. The Cornell method does exactly this. Not only do learners benefit from the creation of questions, but they are then able to self-test by using them – win win!


I’m certainly going to be trialling this approach to note taking with my learners in the coming months, why not do the same with yours? Let’s make note taking more meaningful and use the methods that are likely to be more effective for learning.