Understanding Moderation 17


 

Moderation ensures learners are assessed in a consistent, accurate and well-designed manner. It ensures all assessors who assess a particular unit standard or qualification, are using comparable assessment methods and are making similar and consistent judgments about learners’ performance. 

The unit standards used for training set out the specific details of assessment against the various specific outcomes. These assessment criteria must be considered and used in the moderation process. This will apply to RPL assessment as well as to learners with special needs and who experience barriers to learning. 

Moderation is intended to cover the following 

  1. Assessment instruments, assessment design and methodology as well as the assessment records, the assessment decisions and the reporting and feedback mechanisms. 
  2. Moderation of assessments involving a variety of assessment techniques, such as work samples, simulations, role-plays, written items, oral, portfolios and projects. 
  3. Moderation activities include pre-assessment interactions with assessors, interactions during assessment and post-assessment interactions. 

Moderation within the context of the NQF is a means for professional interaction with, and improvement of practitioners, to continuously improve the quality of assessment. 

The main functions of a moderation system are

  • To verify that assessments are fair, valid, reliable and practicable
  • To identify the need to redesign assessments if required
  • To provide an appeals procedure for dissatisfied learners
  • To evaluate the performance of assessors
  • To provide procedures for the de-registration of unsatisfactory assessors
  • To provide feedback to the NSB’s on unit standards and qualifications

How does Moderation occur? 

Moderation systems combine external and internal moderation. Both external and internal systems must ensure all assessors produce assessments that are credible, fair, valid, reliable and practicable. 

Internal moderation

Internal moderation ensures assessments conducted by a single learning provider, are consistent, accurate and well designed. 

The three main stages to internal moderation are

  • Design

The choice and design of assessment methods and instruments are appropriate to the unit standard and qualifications being assessed.

  • Implementation

The assessment is appropriately conducted and matches the specifications of unit standards and qualifications. This includes ensuring the appropriate arrangements have been made and there are regular discussions among assessors. 

  • Review

Any lessons learnt from the two previous stages are considered and the necessary changes are made. 

Accredited providers should have individuals that manage their internal moderation systems. 

These internal moderators should 

  • Establish systems to standardise assessment, including the plans for internal moderation
  • Monitor consistency of assessment records
  • Through sampling, check the design of assessment materials for appropriateness before they are used, monitor assessment processes, check candidate’s evidence, check the results and decisions of assessor for consistency
  • Co-ordinate assessor meetings
  • Liaise with external moderators
  • Provide appropriate and necessary support, advice and guidance to assessors. 

Providers should have the capacity to implement an internal moderation system that will facilitate and ensure these activities will be carried out effectively and efficiently. 

The internal moderators in learning institutions should be experienced assessors who other assessors have confidence in. They should also have undergone training as moderators and have knowledge of the learning area. 

External moderation 

External moderation is a means of ensuring two or more providers delivering programmes on the same unit standards and/or qualification, are assessing in a well-designed manner, are consistent and maintain a specific standard. 

External moderation involves 

  • Checking the systems required to support the provision of learning programmes within an institution or learning site are appropriate and working effectively
  • Providing advice and guidance to providers and assessors
  • Maintaining a standard over a group of providers
  • Checking all staff involved in assessment are appropriately qualified and experienced
  • Checking the credibility of assessment methods and instruments
  • Checking internal moderation systems
  • Through sampling, monitoring and observing the assessment processes and learners’ evidence to ensure consistency
  • Checking assessors’ decisions. Individuals who will be external moderators should be experienced, know the learning area well, have undergone training for moderation, and have credibility among assessors and within their area of knowledge and expertise. High levels of personal and interpersonal skills are also required. 

THE PROCESS OF MODERATION

  • Moderator is requested to conduct moderation
  • Compile a pre-assessment meeting agenda and checklist
  • Compile moderation planning document and checklist
  • Confirm date, time and venue for meeting with assessor
  • Conduct meeting and keep minutes of meeting
  • Gather required information such as Unit Standard, Assessment instrument and marking memo
  • Compile a post moderation checklist
  • Acquire the agreed % of completed assessments and assessors reports
  • Conduct moderation
  • Complete post moderation checklist
  • Review of moderation process
  • Provide moderation feedback to all parties 

SCOPE OF MODERATION

All moderation must take place in accordance with the scope of moderation as outlined in the relevant unit standard and/or qualification. 

MODERATOR EXPERTISE 

Registered moderators should be 

  • Proficient in assessment practices at the level of the subject-matter of the assessment
  • Registered as assessors
  • Have a broad level of expertise in the subject matter and in-depth expertise in assessments
  • Have undergone training and demonstrated competence in moderation 

As a qualified Assessor a moderator should be familiar with the assessment process. 

The moderators effectiveness will depend to a large extent on his/her ability to ensure compliance by the assessor with the assessment criteria as contained in the unit standard together with a willingness to follow assessment and moderation policies and procedures. 

Assessment and moderation are essential to the effectiveness of any learning. Any action taken by an assessor or moderator that detracts in any way from the experience, the fairness of assessment, the adherence to the specific outcomes and related assessment criteria is a very serious breach of responsibility. 

Moderators should be in a position to 

  • Consider and address issues related to the assessment instrument in terms of design, activities and related documents
  • Moderate assessments taking into consideration candidates with special needs, or candidates applying for RPL assessment.
  • Gather and consider evidence for on-site and off-site moderation
  • Make decisions whether to uphold an assessors decision or not uphold the decision
Share on Social Media

Leave a comment

17 thoughts on “Understanding Moderation

  • Des Squire Post author

    Hi Sharon

    That I would say is not acceptable for a variety of reasons. Most importantly is that moderation follows assessment and cannot be conducted prior to assessment. Something amiss here and definitely grounds for complaint and appeal.

  • SHARON FLYNN

    Hi, can you give me some clarification how does this fit into moderation? Provider sends their internal moderator as a surprise visit to a workshop, asks the facilitator to leave the class and questions the learners about the performance, delivering etc of the facilitator.

  • Des Squire Post author

    Hi Elna, Unfortunately this is the case and I agree with you 100%. Unfortunately some SETA Verifiers fail to check on the assessment and Moderation process resulting in the failures you refer to. Some providers are also to blame in that they do not know or care to adhere to what is required. All I can say is shame on them – this is what gives our industry a bad name. 

    Thank you Wilma and Lance for your kind comments – I just hope my blogs assist others and helps to root out the unscrupulous.  

  • Elna van Jaarsveld

    Excellent, informative post!  However, disappointing experiences from my past: somehow certain training providers manage to “skip” the moderation process.  I am an experienced assessor and moderator and experienced that some of the work I assessed was not physically moderated.  I do not understand how they managed to get their qualifications acknowledged through the relevant SETAs.  I tried numerous times to rectify the situation but just lost my job.  Maybe it is due to a lack of knowledge on the side of the client (large government departments in some cases).  I am extremely passionate about development and growth of people with an excellent success record.  Currenlty I am disappointed in the SAQA processes while I believed it(structured education) could work so well to address a lot of the chalenges in South Africa.

  • Marie Smith

    Well written post, Des. I agree that moderation should be a rather painless process, especially if moderation of design took place at the beginning of the process and competent assessors with subject-matter expertise are being used.

  • Henk Cloete

    My comments, and purely my own, is in agreement with Andrew…

    The current system of moderation, whether internal or external seeks to ensure that the provider adheres to the requirements as “sold” to the respective ETQA when applying for programme approval or accreditation. These “pre-approved programmes” then consists out of prescribed packages that the Assessor then just need to “populate”. (The new 115753 clearly indicates that pre-designed guides must be used and Assessors will not be expected to design assessments as such.) Naturally the learner must supply the evidence to “fit” into these “pre-approved” tools. All of us are more than familiar with the duration, which fortunately differ from SETA to SETA, it takes to get to implementation stage.

    The above then would theoretically lead to comparable assessment methods that would make similar and consistent judgments about learners’ performance a “given.” This system limits the freedom and creativity of Assessors, especially when dealing with learners that have special needs that do not fall within the ambit of the pre-approved programme. Designers does not necessarily cover all the types of contingencies and special needs that may arise!

    This process unfortunately leads to very few providers that will ever dare to make changes without prior approval to assessments, as the threat of deregistration is very real. Unfortunately the learner with the special need then is at the short end.

    Based on the current system, moderation then actually just would need to cover the following:
    Implementation
    Did the provider and the Assessor use the pre-approved programme and assessments?
    Evaluation
    To check if the Assessors actually did complete the sections as set out in the Guide.
    Review
    Monitor consistency of assessment records
    Co-ordinate assessor meetings
    Provide appropriate and necessary support, advice and guidance to assessors.
    Feedback
    To provide feedback to the QC’s on unit standards and qualifications – Never heard of any feedback that was actually implemented…

    External moderation is just an “unbiased” means of ensuring that providers comply with the pre-approved programmes that they have on their scope. We have had SETA moderators that have conducted external moderation / verification whilst only trained as Assessors?

    The current system has evolved into a total paper war and less guidance and training with the providers biggest, and only client, the learner…

  • Johan Bouwer

    Moderation is always a very important part of the skills development process, either in a full qualification or skills program. You just get to many training providers who takes chances or shortcuts during the training. The moderator must be there to assess the process as well as the assessor.

  • Walter Donaldson

    As Moderation is actually ‘Assessing the Assessor’, Moderators should have access, first and foremost, to the CV of the Assessor so that the Moderator can review and assess the experience and qualifications of the Assessor. I’ll not go into too much detail but some of the Assessments that I have seen over the years is not up to standard.

    Let’s not hammer the process too much, let’s get the correct and competent Facilitator and Assessors onto the POEs which, as Des alludes to, should be quite a straightforward (but thorough) exercise. 

  • Victoria Siphiwe Mamvura-Gava

    Hi Des

    I think it is important that refresher meetings/workshops are arranged every so often for assessors and moderators to keep us updated. Surely one should not assessor or moderator an area they have no expertise, this is a basic principle of any professional. Refreshers are good for the benefit of everyone, the providers, the students and the assessors and moderators. Thanks Des for your posting this is another refresher, keep up with the good.

  • Des Squire Post author

    Hi Andrew

    I suggest you change your moderator, for the simple reason moderation should be relatively painless.

    The purpose is well intentioned and from experience is essential as many of the assessments I have seen are far from satisfactory and the assessment process is being abused and badly managed. it is quite frightening how many assessors are assessing and are not qualified or registered to do so. Many assessors and moderators are under the impression that once they have undergone training they can assess or moderate anything they choose as do many HR and training practitioners. 

  • Des Squire Post author

    Hi Matt,

    A moderator must first qualify as an assessor and only then can they complete the training to qualify as a moderator. Once qualified they can register as constituent assessors or moderators with specific SETAs for specific units standards or qualifications.

  • Andrew Friedemann

    All this is good, but when does moderation become yet another unnecessary exercise just to meet bureaucratic ‘processes’? Many assessment we conduct are tiny, less then a few credits worth, and the moderation process is often a bigger task then the assessment. I realise that only a % need to be done, but if a training program and all its materials have been already checked and approved, surely there is no reason to re look at everything after assessment. Rather then Moderation, there needs to be a siimple admin check to ensure all the right materials and processes were followed. I can accept moderation for full qualifications, but for some skills programmes it is just pushing the cost out of reach of those who need the training most.