Looking to create more value in your exam programme? A programme evaluation can be completed at any stage of the exam development process. For those starting from scratch, it provides an invaluable opportunity to build a solid foundation for your exam programme. And if you’re looking to grow your programme, this process enables a period of reflection to ensure a test still serves its main purpose: assessing a desired set of skills in a fair, valid, reliable, and secure way.
Why complete a programme evaluation?
- Maintain a defensible and reliable exam
- Ensure that exam content reflects the intended purpose of the exam
- Provide investment rationalisation to stakeholders
- Support an accreditation application
When you’re considering making changes to your exam, it is vital to understand the nuances of your programme, such as the strength of your candidate volume or the depth of your item bank, in order to maximise the benefits of a new design.
Delivering one test form to 50 people a year is very different from administering five exam forms to 14,000 people, which is also very different from delivering an exam with a more complex item-selection design, such as adaptive or LOFT (linear-on-the-fly testing). Completing a programme evaluation allows you to understand how the item development and psychometric processes would need to evolve to support a larger exam programme. For example, if you deliver a paper-based exam one day a year and want to deliver the exam in a computer-based testing format that leverages on-demand delivery dates, a formal evaluation can identify (1) the psychometric techniques that would enhance the security and comparability of scores across multiple exam forms, and (2) the item development processes that support those psychometric techniques.
Similar to how you would do a home inspection or schedule a doctor’s appointment, it is just as valuable to complete an evaluation as you build your programme as it is to routinely re-evaluate your long-standing programme. Everything might be running smoothly, but evaluating it gives you the transparency to identify any weak points that might need an adjustment in the future.
So what exactly is involved in a programme evaluation?
- Contact your Pearson VUE program manager to arrange a meeting with our leading psychometricians.
- Describe the exam programme and your goals.
- Provide written documentation regarding your processes (item inventory and development, non-disclosure agreements, and style guide, for example).
- Our subject matter experts will review and report on potential improvements or suggestions.
Whether you currently run a computer-based exam or you are thinking about transitioning from paper and pencil, there is tangible value in integrating programme evaluations into your process to ensure that your greatest asset — your exam — is performing how you expect it to. Put your organisation’s best foot forward when certifying the future leaders of the next generation.