Moderation

The fundamental purpose of moderation is to ensure valid and reliable assessment information and that judgments of student progress and achievement are consistent. Moderation helps ensure that you and your colleagues recognise evidence in equivalent ways and confirms your understandings about students’ work. This improves the teacher’s ability to respond appropriately, in terms of: 

• next steps for teaching and learning 
• curriculum design and development. 

Making valid, reliable, and consistent decisions across different points in time is important for reporting on student progress, deciding school targets and resourcing, or comparing cohort data with historical information. 

Moderation improves validity  

Validity means ensuring the assessment task accurately assesses what you want it to assess. Achieving validity requires us to test the task and criteria:  

• What was the intended learning outcome?   
• Do the task and the criteria for success align with that outcome?  
• Are they fair and transparent?  

Moderation improves reliability  

Reliability is ensuring the same results can be repeated over time and in other assessments. It requires that different assessors use the same task, tools, and supporting documents to reach the same judgments. To achieve reliability, it is necessary to achieve consistency in both the descriptions of progress (in tools, such as rubrics) and in the practice of those doing the assessing.  

Moderation creates consistency  

Consistent teacher decisions about progress and achievement are necessary: 

• over time – when a teacher views the same data at different times or in different contexts, they make the same judgment about what it means 
• against progress indicators – the same levels and indicators are applied equally across different types of data 
• by a teacher – across students 
• between teachers – within the same school and different schools 
• between teaching phases of learning. 

All assessment decisions should be moderated if they involve elements of teacher judgment about curriculum expectations. These could include: 

• running records 
• writing assessments 
• numeracy assessments 
• Curriculum Progress Tools – LPF and PaCT
• open-answer responses in common tests internal assessments in NCEA 
• assessment processes and administration.

Some moderation processes are conducted more informally than others. Your school and Kāhui Ako will make decisions based on your needs. The essence is that there is common understanding of how to assess progress and achievement within and across schools. To hold this understanding steady across place and time, maintain up-to-date records that provide a reference point. 

Moderation is a knowledge-building process. When you engage in moderation, you combine your professional content and pedagogical knowledge with your social, cultural, and contextual knowledge to make interpretations and judgments. Individual teachers have different beliefs, understandings, expectations about, and judgments of, student learning. The process of moderation helps to grow the curriculum and assessment knowledge and self-review skills of all who participate. 

The benefits of moderation extend to people at all levels of a school community – ākonga and leaders, as well as teachers. They also extend to your partners in the wider community, including parents and whānau, giving them confidence in the quality of the assessment information brought to learning conversations about learning and progress.  

The benefits of involving students in moderation  

• Ākonga can participate in selecting evidence (e.g., samples of work) that best demonstrate the intended learning outcomes. 
• Ākonga develop their understanding of the desired outcomes and success criteria. 
• Teachers and ākonga have shared expectations of learning.  
• Teachers and  ākonga are clear about what has been learnt and what is yet to be achieved. 
• Both teachers and  ākonga become more assessment literate. 
• Ākonga are better able to engage in self- and peer-assessment as they develop the ability to make judgments. 
• Ākonga feel greater confidence in teacher judgments. 
• There is greater transparency in the assessment process. 

The benefits of moderation for teachers  

• When teachers’ expectations and judgments are aligned with curriculum expectations or progressions, teaching and learning improves.  
• Teachers deepen their understandings about learning content, levels, and progressions. 
• Teachers share their expectations and understandings about learning content, levels, and progressions. 
• The quality of assessment improves.  
• Shared understandings result in greater consistency of judgments. 
• Parents, whānau, and others are assured that teachers' interpretations of students’ progress and achievement are in line with those of other professionals. 

The benefits of moderation for school leaders and boards  

• School leaders and board members feel greater confidence in teachers’ judgments and are assured they are consistent within and across schools. 
• Dependable information can be confidently discussed with partners, including parents, whānau, mana whenua, and others with an interest in ākonga achievement.  
• School leaders and board members become more assessment literate. 
• The knowledge that reliable information is used for teaching and learning decisions helps when communicating with other professional agencies. 
• Dependable achievement information influences strategic directions, including target setting, budget allocation and professional development planning. 

The benefits of moderation for Kāhui Ako  

• The range of schools represented means that teachers can work to define and understand the progressions of learning from year 1 (or before) through to year 13. Teachers across the year levels should be involved in moderation where applicable. 
• Transitions are smoother when schools and other learning institutions have faith in the quality of information shared with them. Develop shared understandings about moderation across Kāhui Ako and other networks.

The moderation process is interwoven with the principles and practice of assessment for learning. It supports improvement in assessment practices and assessment literacy. System and individual teacher decisions can be made with increased confidence. Reliability, validity, and consistency within the process are enhanced, so progress decisions are defensible. The recording of dependable information means it can be used for a variety of teaching, learning, and reporting purposes. 

Examples of where moderation can occur include: 

In teacher-ākonga learning conversations: 

• teachers and ākonga use data and evidence to discuss their interpretations of achievement criteria  
• teachers and ākonga compare samples of work with exemplars 
• teachers and ākonga clarify current skills, knowledge, and understanding; past improvements; and future learning goals 
• ākonga receive dependable achievement information on which to act. 

In teacher-teacher learning conversations: 

• teachers learn from each other, so curriculum and pedagogical content knowledge improves 
• professional learning needs can be identified when analysing progress and achievement data or learning evidence through the moderation process 
• teaching and learning programmes can be adjusted to meet ākonga learning needs 
• individual and collective ākonga achievement trends become clearer. 

This video expands on some implications of moderation for classroom practice. 

In partnership conversations: 

• evidence of learning can be confidently shared 
• reliable information is used to make teaching and learning decisions, which helps when communicating with other professional agencies 
• dependable information can be discussed with parents and whānau 
• dependable achievement information influences strategic decisions, including budget allocation and professional development planning. 

Moderation is most effective when conducted in a spirit of professional learning and quality improvement; participants should expect some dissonance. They need to have appropriate knowledge of the content area, assessment practices and policies, and procedures.  

Moderation is most effective when: 

• it is carried out regularly 
• selected or designed assessment tasks are appropriate and aligned to actual learning 
• it leads to improved learning and assessment 
• it involves as many as possible, so that all gain an understanding of the relevant progressions of learning 
• moderators from outside the school (e.g., facilitators, invited teachers from other schools) are periodically involved to give independent feedback 
• equivalent assessments are agreed, when desired, for cross-class or cross-school comparisons (e.g., Kāhui Ako for professional development purposes). 

Moderation processes are most effective when they are built on a culture of professional dialogue, support, and risk taking. Ensure roles are clearly defined for leaders, teachers, and ākonga, including a moderation leader  

Getting the moderation process right is important. To assist with this, we have separated the moderation process into three sections.

Process: planning for moderation

Initial decisions  

Schools make important decisions prior to engaging in moderation.  

Leadership role, prior to moderation  

If you have responsibility for leading moderation, you could use this checklist to ensure the moderation time is purposeful and achieves its aims.  

Meeting for moderation

Goals for a moderation session  

• Achieve shared understanding and consistency of use of assessment criteria and language. 
• Identify similarities and differences in judgments. 
• Resolve any differences. 
• Achieve consistency in people’s judgments. 

Suggested format for a moderation session 

1. All teachers look at the same piece/s of work. 
2. Minutes/conclusions from the session are distributed to all teachers. 
3. A leader records suggestions for improving how future moderation sessions are run. 
4. A leader records any changes in judgments arising from the discussion. 
5. The group works towards consensus about the level of achievement to be assigned for each key feature. 
6. All teachers have input to the discussion. 
7. Teachers provide evidence and reasons for their judgments, looking at the areas of similarity and difference. Time spent exploring understanding of learning progressions or the language used in the descriptors will lead to greater shared understanding of the assessment criteria. 
8. The group reviews its judgments and identifies areas of greatest similarity and difference. 
9. A leader or scribe records everyone’s judgments. 
10. All teachers share their judgments. 
11. Teachers record their judgments on a recording sheet.  
12. Teachers assess the work against the success criteria or indicators. 
13. Everyone reads the whole script, text, collection of evidence through without marking, commenting, or discussing it. 

Tips for getting the best out of your meeting  

Use differences as an opportunity to deepen the knowledge base of the group

• Ask questions to clarify thinking and understanding of language. 
• Explore solutions – find agreement based on evidence and reasoned discussion. 
• Be prepared to adapt your thinking and adjust your judgment after listening to the informed ideas of others.
• Identify where more knowledge or resources are needed, and where processes could be modified. 
• Get the views of all teachers. Inquire into each teacher’s thinking, encouraging them to share the reasons for their judgments.   

Keep discussions focused and useful  

• At the start of the process, expect and tolerate disagreement. 
• Share your thinking in relation to evidence of key features, not just ‘gut feelings’. 
• Discuss reasons for differences, such as different expectations, or interpretations of language. 
• Check teachers’ understanding of the language used and reach shared clarity. Use professional resources to do this. 
• Keep focused on the key features. 
• Ensure all teachers’ views are heard. 
• Treat it as a learning exercise, not a check to see if judgments are ‘right’ or ‘wrong’. 

Check for bias  

Assessment that relies on a significant degree of teacher judgment is primarily subjective. It can be useful to examine bias with teachers as “bias can result, unconsciously, from prior dealings with students based on attitude, behaviour, gender, race or disability.” (Adie, 2008).  

Common biases in assessing ākonga work include: 

• making assumptions based on the engagement, behaviour, socio-economic background, or the race of the ākonga. 
• considering longer texts to be more worthy than shorter ones 
• considering neater handwriting to be more worthy than untidy writing 
• using internalised, unstated standards that individual teachers have developed over time, instead of explicit, agreed criteria based on the curriculum 
• giving students the benefit of the doubt in order to be ‘fair’ rather than being guided by what the evidence shows 
• judging work based on teachers’ beliefs about what students deserve or their judgments about student effort. 

Reviewing moderation processes  

To create effective moderation systems, leaders ensure that assessment cycles incorporate regular moderation. They provide regular opportunities for teachers to share their interpretations and understandings of criteria, and actively support the moderation process. 

Reviewing your processes routinely ensure they are fit for your learners and community. The goal is to develop consistent and cohesive policies and procedures for moderation and ensure sustainability of practice. It is a good idea to have assessment practices and moderation processes recorded in sufficient detail. 

Useful discussion questions about moderation: 

• How high is the level of comparability across teachers? 
• Is the judgment of each ākonga sample consistent and fair? 
• What did the moderation process reveal to me/us about my/our knowledge of the curriculum and progressions of learning, or assessment? 
• Does our school already have annotated exemplars available for use as references? Do we need to establish a collection? 
• What does moderation show that I/we might do differently in my/our teaching to help students’ learning and achievement? 
• How can our school improve its moderation process? Where can we go for support? 
• How involved are the ākonga in the moderation process? How can we support them to moderate their own work within their classrooms? 
• What does moderation look like when it is done in relationship to progressions of progress created by our community? 
• Who needs to be involved in moderation to ensure its reliability and validity in terms of progress within our local curriculum? 
• What further professional learning might I/we need? 
• How might our moderation processes be extended to other curriculum areas? 
• How can the information gained from moderation be shared across other areas of the school or across schools in the Kāhui Ako? 
• How consistent are decisions within and across schools? 
• How useful is our recording of the process and how can it be improved? 

Challenges to consistency  

All schools experience variables that challenge the consistency of moderation practice, such as staff and student changes, or changing educational demands. To maintain consistency: 

• Always apply the same standardised criteria. 
• Where nationally standardised criteria or exemplars are available, use these as the external reference point every time you engage in moderation. 
• Augment your school’s approach by adding your own student samples to your exemplar collection to reflect the flavour, context, tikanga, and cultural richness of your community.  
• As moderators change over time, ensure they use the same criteria and associated reference points to guide their decisions. 

Suggestions for attaining greater consistency  

Make judgments based on trustworthy data and evidence 

• Your judgments should be based on adequate evidence of student learning. This means that it should have been interpreted with reference to a trusted framework of curriculum knowledge. 
• The evidence may be visual, written, oral, or a physical construction of learning. For it to be ‘adequate’, you should be sure that the learning is embedded and not just a one-off or fluke occurrence. 

Collect appropriate information before making judgments 

• Decisions about the evidence you collect and how you collect it will depend on your purpose, the type of information needed, and its intended use. 
• For daily teaching and learning purposes, one-off informal judgments might be used. For example, observations during teaching, recorded comments in modelling books, or student self-assessment, such as highlighting indicators on a matrix based on the curriculum. 
• For reporting and accountability purposes, judgments need to be more extensive, formal, consistent, and comparable. 
• The evidence you collect should be fair and appropriate to ensure it promotes consistent and comparable judgments. 

Use assessment information that has been gathered thoughtfully and fairly, using the appropriate tools to make judgments  

Ask questions, such as: 

• Does everyone have a shared understanding of the assessment tools that can be used?  
• If using a standardised tool, are the administration processes the same across all classrooms? 
• Are we giving students sufficient opportunities to demonstrate their current capability? 
• Have we made adjustments when necessary, such as enlarged copies of the assessment task for visually impaired, longer time frames for physically impaired students, choice of topics, or use of heritage languages? 

Judgments should be consistent 

• Over time: How do you know and ensure you reach consistent judgments of students’ work throughout the year? (E.g., this could be by collecting the same evidence at the same time, year by year). 
• Across students: Do you use the same evidence to make judgments about different students? How do your judgments vary across gender, ethnicity, or individual traits (behavioural, learning etc)? Should they vary? 
• Across classes or schools: Do you use the same evidence to make judgments about different classes or schools? How consistent are your judgments from year to year, across classes or schools? How could you find out? 

Be attuned to the cultural and social dimensions of the assessment process 

• Assessment is not a simple matching exercise that occurs between a work sample and standards of achievement. 
• Teachers use social, cultural, and contextual knowledge in forming judgments of student work. Be open to examining your beliefs and possible preconceptions. 

Assessment is a complex task that is grounded in the social and cultural experiences of those involved (Adie, 2008).