Another key element in designing competency-based assessments is deciding what evidence needs be collecting to meet competency requirements. The key things to be considered are: the type of evidence, the quality of the evidence and how it will be stored.
What sort of evidence is collected?
Evidence collected may be:
- direct – such as observation of workplace performance;
- indirect – such as formal testing of knowledge; or
- supplementary – such as references and attestations from employers.
Evidence is used by an assessor to make a judgement about whether a learner is competent. When designing assessments is important to consider and determine what and how much evidence is required to make the assessment judgement. Qualification documents and training packages published by qualification and assessment authorities usually provide guidance on the types of evidence required. Further advice may be gained through industry consultation or involving industry partitioners in the assessment design process.
Evidence gathering methods commonly used may include, but are not limited to:
- projects
- workplace assignments
- workplace performance
- documentation
- demonstration in person or video
- questioning and professional conversations
- role play
- simulation
- presentations in person or video
- written tests
- portfolio
- third party reports
Types of evidence
There are four different types of evidence to consider in the context of assessing competency in an online and workplace environment:
- Real work / real time evidence can be collected by practical observation in the workplace or technologies such as video and image sharing, digital stories, and video streaming may be used to collect direct evidence of learner performance on real work tasks in real time. This may be accompanied by online self and peer assessments as well as comments from workplace supervisors.
- Simulation and demonstrations whether in person, video recorded or simulated by a computer may be used to collect evidence of performance. Simulations range from simple demonstrations of workplace tasks and role plays, which may be captured on video and reviewed, through to complex applications such as virtual simulations in which multiple users can interact together or with objects, characters or interactive environments.
- Questioning is often used as supporting evidence of knowledge and understanding. Online quizzes created either within or outside an LMS may be used to test and record that. Generic programs, such as Skype, ZOOM, Microsoft Teams, or specialist applications, such as virtual classrooms, may be used for questioning, interviews and discussions if not possible to conduct at the workplace or in person.
- E-portfolios are online collections of reflections and digital artefacts, such as documents, images, videos that may be assembled by learners to demonstrate and record their learning and skill development over time.
Quality of evidence
Where assessments are online or at the workplace they need to provide quality evidence to support assessment decision making.
To be of acceptable quality evidence need to be: valid, sufficient, authentic and current.
Validity is concerned with the extent to which an assessment decision about a learner (eg competent/not yet competent), based on the evidence of learner performance, is justified. It requires determining conditions that could impact on the truthfulness of the decision and feeding back into the assessment process to reduce errors when making a competency judgment.
Validity requires the evidence to assure the assessor that the learner has the skills, knowledge and attributes described in the module or unit of competency/ outcome and associated assessment requirements.
For example, if assessing learner’s competence in understanding and using technical drawings as part of a carpentry programme the assessment tasks needs to provide the technical drawings for the learner to take measurements and cut the needed wood for the project, rather than provide a pre cut pieces of wood that the learner works with.
Sufficiency relates to the quality and quantity of evidence assessed. It requires collection of enough appropriate evidence to ensure that all aspects of competency have been satisfied and that competency can be demonstrated repeatedly. Supplementary sources of evidence may be necessary. The specific evidence requirements of each unit of competency can provide advice on sufficiency.
Sufficiency requires the assessor to be assured that the quality, quantity and relevance of the assessment evidence enable a judgement to be made of the learner’s competency.
For example, if we are assessing a learner ability to diagnose and repair faults in mobile phones the assessment needs to involve making repairs of different faults (let’s say at least five different hardware parts need to be replaced and two software issues resolved) in mobile phones with two different operating systems (e.g. Android and iPhone). If only a couple of faults are diagnosed and repaired in iPhones that will not be sufficient evidence of the learner’s competency to work with the main operating systems.
Authenticity relates to how an assessor is assured that the evidence presented for assessment is the learner’s own work.
Authenticity requires the assessor to be assured that the evidence presented for assessment is the learner’s own work.
For example, if a learner is providing the business records they have been keeping for a workshop as evidence of record keeping the records need to be verified by the workshop owner, in assurance that they have been produced by the learner.
Currency relates to how old is the evidence presented by a learner to demonstrate that they are still competent. Competency requires demonstration of current performance, so the evidence collected must be from either the present or the very recent past.
Currency requires the assessor to be assured that the assessment evidence demonstrates current competency. This requires the assessment evidence to be from the present or the very recent past.
For example, first aid evidence may be acceptable within six months to a year.
Where and how assessment evidence is stored
Last but not least you need to find out what assessment items need to be kept for each learner, for how long and under what conditions. The best way to find that out is to refer to your institutional policy. If there is no such policy in place it is best to check what the rules and requirements are with the qualification and assessment authority.
Another key element in designing competency-based assessments is deciding what evidence needs be collecting to meet competency requirements. The key things to be considered are: the type of evidence, the quality of the evidence and how it will be stored.
What sort of evidence is collected?
Evidence collected may be:
Evidence is used by an assessor to make a judgement about whether a learner is competent. When designing assessments is important to consider and determine what and how much evidence is required to make the assessment judgement. Qualification documents and training packages published by qualification and assessment authorities usually provide guidance on the types of evidence required. Further advice may be gained through industry consultation or involving industry partitioners in the assessment design process.
Evidence gathering methods commonly used may include, but are not limited to:
Types of evidence
There are four different types of evidence to consider in the context of assessing competency in an online and workplace environment:
Quality of evidence
Where assessments are online or at the workplace they need to provide quality evidence to support assessment decision making.
To be of acceptable quality evidence need to be: valid, sufficient, authentic and current.
Validity is concerned with the extent to which an assessment decision about a learner (eg competent/not yet competent), based on the evidence of learner performance, is justified. It requires determining conditions that could impact on the truthfulness of the decision and feeding back into the assessment process to reduce errors when making a competency judgment.
Validity requires the evidence to assure the assessor that the learner has the skills, knowledge and attributes described in the module or unit of competency/ outcome and associated assessment requirements.
For example, if assessing learner’s competence in understanding and using technical drawings as part of a carpentry programme the assessment tasks needs to provide the technical drawings for the learner to take measurements and cut the needed wood for the project, rather than provide a pre cut pieces of wood that the learner works with.
Sufficiency relates to the quality and quantity of evidence assessed. It requires collection of enough appropriate evidence to ensure that all aspects of competency have been satisfied and that competency can be demonstrated repeatedly. Supplementary sources of evidence may be necessary. The specific evidence requirements of each unit of competency can provide advice on sufficiency.
Sufficiency requires the assessor to be assured that the quality, quantity and relevance of the assessment evidence enable a judgement to be made of the learner’s competency.
For example, if we are assessing a learner ability to diagnose and repair faults in mobile phones the assessment needs to involve making repairs of different faults (let’s say at least five different hardware parts need to be replaced and two software issues resolved) in mobile phones with two different operating systems (e.g. Android and iPhone). If only a couple of faults are diagnosed and repaired in iPhones that will not be sufficient evidence of the learner’s competency to work with the main operating systems.
Authenticity relates to how an assessor is assured that the evidence presented for assessment is the learner’s own work.
Authenticity requires the assessor to be assured that the evidence presented for assessment is the learner’s own work.
For example, if a learner is providing the business records they have been keeping for a workshop as evidence of record keeping the records need to be verified by the workshop owner, in assurance that they have been produced by the learner.
Currency relates to how old is the evidence presented by a learner to demonstrate that they are still competent. Competency requires demonstration of current performance, so the evidence collected must be from either the present or the very recent past.
Currency requires the assessor to be assured that the assessment evidence demonstrates current competency. This requires the assessment evidence to be from the present or the very recent past.
For example, first aid evidence may be acceptable within six months to a year.
Where and how assessment evidence is stored
Last but not least you need to find out what assessment items need to be kept for each learner, for how long and under what conditions. The best way to find that out is to refer to your institutional policy. If there is no such policy in place it is best to check what the rules and requirements are with the qualification and assessment authority.
Course developer
Partner