Industry Research Report: Test Development


Prometric commissioned a survey of test development professionals responsible for managing their organization’s certification process to understand how professional testing and certification organizations meet their needs and the areas where they experience both success and difficulty. Telephone interviews were conducted with 82 individuals between February 25 and March 18, 2011. The survey instrument included a mix of constrained response and open-ended questions. Respondents represented organizations of varying size and from a broad range of industries and professions, including accounting, construction, education, healthcare, information technology, law and telecommunications.

This Prometric-sponsored report presents data from a nationwide survey of professionals directly involved in – or managing – test content development. The data presented in the paper show test development processes that rely considerably on informal, improvised mechanisms and practices, particularly for item writing and job analysis. Paradoxically, these are areas of the test development process that professional organizations tend to manage in-house, rather than utilize external resources, according to the Prometric survey. Additionally, the data show that relatively few testing and certification organizations have taken advantage of platform-neutral, industry-standard test formats that facilitate collaboration in item development and exchange of test material and data across systems.

As a companion to our 2010 Assessment Alliance survey on Test Development, the detailed report provides an overview of survey responses related to five critical areas of the test development process:

  1. Item Development
  2. Test Design and Construction
  3. Job Analysis
  4. Item Analysis and Psychometrics
  5. Test Publishing

Subsequent reports will provide closer examination of each of these components of developing and delivering effective professional credentialing exams, with specific focus on areas that are particularly challenging to organizations that offer credentialing programs. Download the paper now!