AHIMA’s Commission on Certification for Health Informatics and Information Management (CCHIIM) Test Development Process Jo Santos, RHIA Senior Manager, Test Administration

© 2014 Mission & Vision Mission: Through certification, CCHIIM assures the competency of professionals practicing health informatics and information management worldwide. Vision: Professional excellence in health informatics and information management through certification.

© 2014 Values Application of evidence-based best practices for certification; Validation of workforce competence through professional certification; Commitment to ongoing professional development, lifelong learning and workforce excellence, and Recognition of CCHIIM-certified professionals' role in maintaining and enhancing quality health information for the safety of the public and the improvement of health care.

© 2014 CCHIIM Serves the public by establishing, implementing and enforcing standards and procedures for certification and recertification of health informatics and information management (HIIM) professionals.

© 2014 Membership Societies growth and relevance through advocacy and community AHIMA – CEE & Foundation Accreditation preparation through standardized education CAHIIM Certification entry, retention, and development through competency validation and maintenance CCHIIM Collaboration

© 2014 CCHIIM Volunteer Structure CCHIIM Appeals Committee Nominating Committee Exam Security Committee Work Groups Exam Development Committees (9) Job Analysis Task Force Item Writers (~200)

© 2014 At- Large Commissioners Representative Commissioners Public Member

© 2014 Appeals Committee Exam Eligibility Exam Performance RecertificationEthics Appeals

© 2014 Nominating Committee The responsibilities of the CCHIIM Nominating Committee shall be to propose at least two (2) qualified candidates for each At-large Commissioner position vacancy, in accordance with the policy and procedures.

© 2014 Exam Security Committee All certification exam content is secure Only those individuals truly eligible to take an exam do so No candidate has an unfair advantage over another Risks are identified and addressed appropriately

© 2014 Exam Development Committee Develop and maintain examinations Provide expertise on setting the examination cut score Provide expertise on the methodology of scoring the examination Ensure content validity Monitor and adhere to the examination blueprint and specifications

© 2014 Item Writers Item Writing Workshops (2)Eligible for honorariumEarn CEUsConfidentiality agreementNetwork with your peers

© 2014 Job Analysis Task Force The task force includes HIM professionals who: Hold the credential for which the job analysis will be completed Represent various geographic regions and diverse job roles and settings.

© 2014 AHIMA Certifications

Test Development Cycle

Needs Analysis Determine if the credential is needed Job Analysis Conduct job analysis to come up with task list and domains Create survey Determine survey administration method, format, and collect survey participant info Administer survey Administer survey for 1 month Analyze Survey Results Analyze survey results to determine tasks, number of items, and distribution of items to test on

Test Development Cycle - continued Specifications Call Present to JA group to determine Outline specifications Item Writing Hold item writing meetings to collect enough items to build the exam Item Review First reviewed by staff for proper format; Group Reviews (SME's) for Content Build exam forms Exam forms are built based on exam specifications Form Review Forms are reviewed by SMEs to ensure the appropriate distribution of items and that there are no enemy items on a form

Test Development Cycle - continued Forms finalized for publication Once finalized the forms are packaged and sent to delivery vendor for QA Exam Approval Exams are approved for publication and beta is launched Analysis and Standard setting Once enough results are collected (~1 mo) the results are analyzed and standard setting is conducted and cut score is determined Exams are relaunched (rescore and live versions) Exam scoring data and necessary updates are made to the exam and the live version of the exam is launched (immediate scoring) Item Writing The exam maintenanc e process begins. Every few years (time frame depends on how often job changes) the process will go back to include the job analysis.

Job Analysis Defines practice of a profession Defines tasks practitioners must be able to perform competently – observable Maintain accurate description of a practice Should be reviewed every 3-5 years Provides framework for examination Fair Job related Legally defensible Essential method of demonstrating job relatedness Critical component in keeping certification examination content current Required by psychometric standards

Why Job Analysis? Standards for certification and licensure examinations call for job analysis as the foundation of a testing program – NCCA (National Commission for Certifying Agencies) – ISO 17024: Personnel Certification – Joint standards (AERA/APA/NCME) American Educational Research Association American Psychological Association National Council on Measurement in Education

Job Analysis Task Force Considerations: Job Role Industry Environments Demographics Geographic Representation Work Environment Size Other Stakeholder Groups

Job Analysis Process Flow Call for job analysis task force participants Vet and confirm task force participants Preliminary Conference Calls for Task Force In-person Job Analysis Task Force Meeting Validation Survey - Pilot Group Full Survey Test Specification Meeting Finalize Job Analysis Report

Examination Framework Domains Sub-Domains (if applicable) Tasks or Competency Statements Knowledge Topics Test Specifications

Different points in measurement

Information Flow

Job Analysis for Exam Maintenance Job analysis Content Outline Test Specifications Test development Evaluate Change in the field

©2015 Questions