When deciding on a developer or a supplier of off-the-shelf SAQA-aligned outcomes-based learning material and assessments, the Training Provider’s dilemma is how to evaluate the potential contractor or supplier.
Some time ago, the question arose of what guidelines can be given to training providers when they have to decide on contracting a designer/developer of outcomes-based learning material and assessments or when they buy ‘off-the-shelf’ material.
The suggestions that follow (which are by no means hard and fast rules or an attempt to be exhaustive) might be useful.
A little bit of homework may save you money and frustration down the line.
Although there is no fool-proof recipe, the following guidelines should be helpful:
- Credibility – the qualifications, experience and subject matter expertise of the person who designed and developed the material. Here, qualifications do not only include ‘technical’ training in the topic, but also training in design and development of outcomes-based material and assessments. A developer with at least a few years of experience as Facilitator, Assessor and Moderator has a firm base for understanding what makes for material that meets education and training standards and expectations – from all points of view. Actual work experience in the field (e.g. supervisor/manager/sales assistant) provides a solid background in understanding the key deliverables for a target group; this will help in the development of material that helps learners acquire the skills they need.
- Reputation – can the developer provide references that give evidence of reliability in terms of delivery?
- Validity and sufficiency of assessments to meet moderation requirements (See the separate post on validity and sufficiency of evidence).
- Assurance. Find out what assurance the supplier is prepared to give, for example:
- Guarantee that material meets/will meet moderation and SETA verification
- Remedial action should it be necessary.
The criteria for assessments are that they should be:
Validity implies the assessment focuses on the requirements laid down in the unit standard, i.e. the assessment is fit for purpose.
Validity therefore requires the assessment to focus on what the specific outcome and related assessment criteria require the learner to be able to do. For example, if the assessment criterion states ‘Explain..’ the assessment should be designed in a manner that requires the learner to list, identify, etc. When the learner is required to ‘Apply..’ the assessment should be designed in a manner that requires the learner to provide evidence that the knowledge (such as ‘using benefits to address customer needs’) has been applied in a work environment. In the example of using benefits to address customer needs, the evidence will have to provide proof of this happening.
Sufficient implies that the evidence collected establishes that all criteria have been met and that performance to the required criteria can be repeated consistently.
In my experience as moderator of learning programme material and assessments, the biggest shortcoming that I have observed in assessments designed by lesser-experienced developers of material and assessments is that there is lack of validity or misinterpretation of the specific outcomes and/or assessment criteria. For example, outcome 3 for US 118029 refers to hygiene practices in a wholesale/retail outlet (i.e. housekeeping and hygiene) and not merely personal hygiene of employees.
To ensure that the material developed meets the assessment criteria, it should be moderated before facilitation – a requirement of all SETAs. Such moderation will not only meet the SETA ETQA Quality Management System requirements, but might also save you the embarrassment and frustration that comes at the moderation stage if your material does not meet the requirements of the unit standard appropriately.