BLOG SERIES: The Value of Pre-Employment Testing – 4-Part Series
Part 4: Competency-Based Testing – Customized Assessments
In some situations a standard, off-the-shelf assessment does not accurately measure the job standards for a particular company or organization. In these cases the organization can customize an assessment specific to their needs. Nocti Business Solutions (NBS) assists organizations that have developed their own standards in creating customized assessments to measure those standards.
New and emerging occupations and requests from clients to customize assessments require NBS to be continuously involved in test development. NBS meets the international standards (ISO 17024) for organizations dealing in personnel credentialing, and meets accepted national standards established by the American Psychological Association (APA), the American Educational Research Association (AERA), and the National Council Measurement in Education (NCME).
If a customized assessment is developed, a Job and Task Analysis is always performed with a team of Subject Matter Experts (SMEs) identified by the organization. The resulting critical core competencies, tasks and steps are then ranked and quantified to create a blueprint for the assessment being developed. Assessments are then developed in accordance with the established blueprint and SME recommendations, ensuring all competencies covered in the job description are addressed.
A customized assessment development can consist of a written(cognitive) or performance (psychomotor) assessment or the company can develop both written and performance assessments based on the same standards identified during the Job and Task Analysis process. The new assessment is then pilot tested utilizing a sample of individuals to ensure items on the written assessment are performing as expected and job criteria on a performance test are appropriate. The results of the pilot are compiled and analyzed. The results of this analysis indicate item difficulty and discrimination, mean, standard deviation, range of scores, reliability index, and the standard error of measurement and the proportion of test takers who gave the correct answer.
A bias review is also performed on all items/jobs in the new test. Each test item/performance job is read and reviewed to verify that language usage, stereotyping and/or representational unfairness biases do not exist.
During a customized assessment development, the final step of the validation process is to set a legally defensible passing score for which to set the benchmark on the assessment. A psychometrically sound standard-setting process is utilized with the SME team to determine the cut score. Pilot test data and demographic information is used as part of the cut score determination. During this process, the SME team also identifies any additional modifications to be made to the assessment items or jobs prior to launching the new assessment.
NBS offers many types of customized reporting. Each report is designed specifically to meet customers’ needs. Customized reports can be designed for states or regions or can be separated by demographics.
By using the processes described above, Nocti Business Solutions can assure organizations that the competency-based assessments they are using have been put through a strict validity process, are reliable, and most importantly, are legally defensible.
Recent Comments