Primary maths diagnostic assessment - making a choice: Multiple choice questions = multiple benefits?

    Published: 10 November 2020
    Multiple choice questions

     

    Multiple choice questions are often looked down on by many teachers and considered to

    be the lowest of the low in testing terms.

    Common criticisms that are hurled at multiple choice questions include:

    1. The answers are easy to spot. Pupils know that the correct answer is there somewhere so they can just work backwards and use elimination to identify it. So do they really know it? If they had been asked the same question, but using a free text response style, would they have been able to get anywhere near the answer?
    2. Pupils guess the answers. Pupils get marks for questions they don’t really know.
    3. They can be harmful. Pupils are exposed to incorrect responses so their misconceptions are re-enforced.

    Many teachers see them as having only one obvious positive:

    1. They are quick to administrate and can be digitised easily.

    So why did the HfL primary maths team go for multiple choice questions when they are so controversial?

    The simplest response is that cognitive learning researchers say that they can be powerful learning events if used correctly (Little, Frickey and  Fung. 2019). Within this blog, I will attempt to add a little more detail to why we went along this route.

    There are two dominant strategies when answering multiple choice questions.

    Firstly, thinking about the possible answer before you look at the alternatives – this tends to be favoured when you are confident about the subject content.

    Otherwise, many of us employ the ‘elimination’ strategy, which involves systematically going through the alternative answers and dismissing unlikely candidates. Are both of these strategies learning neutral? Are you thinking about nothing?

     

    Multiple choice questions

     

    Do you recognise these answers?

    They are extracted from the School Inspection  Framework (2019); the school’s use of assessment:

    Section 185 – When used effectively, assessment helps pupils to embed knowledge and use it fluently and assists teachers in producing clear next steps for pupils. However, assessment is too often carried out in a way that creates unnecessary burdens for staff and pupils. It is therefore important that leaders and teachers understand its limitations and avoid misuse and overuse.
    Section 186 – Inspectors will therefore evaluate how assessment is used in the school to support the teaching of the curriculum, but not substantially increase teachers’ workloads by necessitating too much one-to-one teaching or overly demanding programmes that are almost impossible to deliver without lowering expectations of some pupils.

    Ofsted (2019) School inspection handbook

    Multiple choice questions help pupils to embed knowledge.

    It is well documented that when information is successfully recalled, the actual act of retrieval modifies the memory to make it more recallable in the future (McDermott et al, 2014, Bjork et al 1996), thus moving the memory closer to fluent recall. Moreover, information that is harder to recall, once recalled correctly, has more of a learning boost, so it will be significantly easier to recall next time it is required. 

    When pupils engage with multiple choice questions, they know that the correct answer is there somewhere. This has been shown to reduce self-imposed pressure on the children and allows you to get a more genuine representation of their knowledge at that moment in time.

    To maximise the learning impact that multiple choice questions can have, the response options have to be considered carefully, creating what Little and Bjork (2015) refer to as competitive alternative answers. It is the inclusion of these intelligently crafted options that lift multiple choice tests from simple recall quizzes to actual learning opportunities.

    The best competitive alternatives actually represent plausible answers; they replicate the misconceptions that the pupils may hold themselves, forcing them to think about not just why a is correct, but also why b, c and d are incorrect, replicating the elimination strategy. This filtering and dismissing triggers harder to recall memories and so boosts them.

    Consider these examples from the HfL Primary Maths Diagnostic Tests.

     

    Maths question

     

    Year 3 Spring Reasoning Paper

     

    Maths question

     

    Year 1 Autumn Reasoning Paper

    Mathematics involves children in becoming fluent in volumes of confusable information. With diagnostic questions, we have the opportunity to place these potential confusions side by side, forcing children to discriminate between them. By deeply considering the alternative, believable answers, pupils may recover and stabilise marginal knowledge (Little, 2019).

    For example:

     

    Maths question

     

    Year 4 Summer term Reasoning Paper.

     

    Maths question

     

    Year 1 Summer term Reasoning Paper.

    Multiple choice questions help teachers to produce clear next steps for pupils.

    The HfL Diagnostic Tests are complemented with two additional tools: a planning guide and a digitised mark book.

    These combine to streamline workload and release time to focus on how to use the resultant data to improve outcomes for pupils.

    Obviously, you can identify the errors that pupils are making in free text questions. By closely examining every question that the pupils have got incorrect, you can identify what errors are being made. However, this takes time and specialist knowledge. Multiple choice questions which have competitive alternative answers can remove this time burden. In the HfL diagnostics, the alternative answers have been selected to expose common misconceptions, miscalculations or slippages of knowledge. We have added these additional details to the Planning Guides, explaining why these alternative options have been selected and what error they may represent.

    This provides teachers with the ‘what needs to be fixed’. The next step is how to fix it.

    Multiple choice questions do not substantially increase teachers’ workloads.

    The digital mark books allows teachers to produce forensic diagnostic profiles for the whole class, or through the use of filters, individual pupils or groups in a matter of minutes.

    Using the simple proforma provided, teachers can rapidly develop clear and focused actions.

    • Everyday fluency – Which areas could be maintained or strengthened with inclusion into the 10 minute a day fluency sessions outside of the maths lesson?
    • Whole class input – Which areas need reference back to prior learning before teaching?
    • Small group focus – Who needs more and what of? How will this be arranged and who will deliver it?
    • Individual pupil support – Who needs a 1:1 focus and what on? Focus on high value areas such as place value, number sense and mental methods first. How will this be arranged and who will deliver it?

     

    Table

     

    Snapshot of one of the pages from a Year 4 Diagnostic Mark Book.

     

    Graph

     

    Snapshot of one of the pages from a Year 4 Diagnostic Mark Book with filters activated.

    Pie charts allow teachers to see the distribution of different alternative answers.

     

    Pie chart

     

    The planning guide helps to identify:

    • the mostly likely possible error,
    • which upcoming sequences will be building on this learning,
    • recommended pre-teaching sequences which address the identified error.

     

    Graphic with text

     

    A sample page from a Year 4 Planning Guide

    There is a difference between a mistake and a misconception.

    A misconception occurs when an idea or a concept hasn’t been fully grasped. Time is precious; teachers can’t be deploying valuable resources to fix a gap which is, in reality, just an error. For this reason, ‘further check questions’ have been designed to be used with identified children prior to any possible intervening.

    So to return to the original multiple choice questions:

     

    Multiple choice questions

     

    Correct answer: C

     

    Multiple choice questions

     

    Correct answer: D

    The HfL Primary Maths Diagnostic Test papers come as termly sets for each single year group and mixed year group combinations of Year 1 / 2, Year 3 / 4 and Year 5 / 6.

    Each set includes:

    • An arithmetic paper
    • A reasoning paper
    • A digital markbook
    • A planning guide

    If you would like to find out more about the HfL Diagnostic Assessments, please visit:

    Primary mathematics teaching and learning advisory services


    References

    Bjork, R. (1994). Memory and metamemory considerations in the training of human beings.

    Little, J. L., Frickey, E. A., & Fung, A. K. (2019). The role of retrieval in answering multiple-choice questions. Journal of Experimental Psychology: Learning, Memory, and Cognition, 45(8), 1473–1485. https://doi.org/10.1037/xlm0000638

    Little, J., & Bjork, E. (2015). Optimizing multiple-choice tests as tools for learning. Memory & Cognition, 43, 14-26.

    Little, J., Bjork, E., Bjork, R., & Angello, G. (2012). Multiple-Choice Tests Exonerated, at Least of Some Charges. Psychological Science, 23, 1337 - 1344.

    McDermott, K., Agarwal, P.K., Dantonio, L., Roediger, H., & McDaniel, M. (2014). Both multiple-choice and short-answer quizzes enhance later exam performance in middle and high school classes. Journal of experimental psychology. Applied, 20 1, 3-21 .

    Ofsted (2019) School inspection handbook:

    Handbook for inspecting schools in England under section 5 of the Education Act 2005

    https://www.gov.uk/government/publications/school-inspection-handbook-eif

    Contact details

    Latest blogs

    Receive our latest posts direct to your inbox...