1. ABA Roundtable May 2014

2. IN THE BEGINNING,.... There was nothing.

3. OVERALL EXAM STRATEGY A MOVE TO CONSCIOUSNESS

4. SOME EXAMPLES Format Specifications Purpose Drafting guidelines

5. STRATEGY FOR EXAM DATABASE Make maximum use of available exam information each year

6. TACTICAL EXAMPLES We can now generate an exam in final format from the database This was humongous for us – we had previously used Word documents in preparing exam content for 21 specialty areas It reduced the work for our volunteers

7. MORE TACTICAL EXAMPLES Questions can be cataloged by subject and a unique ID number Questions can be tied to a specific provision in the exam specifications We can track who drafted, graded or reviewed We have authority for the correct answer We can determine if the question needs to be revised We can revise a model answer if needed

8. CONTINUED We can track performance on multiple choice and essay questions We can determine if a question is invalid and adjust scores accordingly in the database We generate a detailed report on multiple choice performance We can easily use different combinations of questions to keep an exam from being too predictable

9. CONTINUED We can better quantify the coverage of the exam so that topics were covered in the proper proportion to their importance in the specialty area We can determine how examinees performed by topic We can much more easily track changes to questions and retire items which are no longer valid