1. “You Can’t Always Get What You Want: Accountability and Limits in Score Reports to Boards” Troy Elliott Association of Social Work Boards 2006 Annual ConferenceAlexandria, Virginia Council on Licensure, Enforcement and Regulation Expect the Unexpected: Are We Clearly Prepared? Censored image of psychometrician in Mick Jagger style tight leather pants

2. Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia Who We Are Association of Social Work Boards (ASWB) 49 states, DC, USVI, 7 provinces Public protection mission Social work licensure examinations—about 25,000 administrations annually Other services: Disciplinary databank, continuing education provider approval, credentials registry, communication, etc.

3. Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia The ASWB Examinations Five categories (Associate, Bachelors, Masters, Advanced Generalist, Clinical) 4 option multiple choice 150 scored, 20 pretest CBT 4 hour time limit Psychometric and administration support through ACT, Inc.

4. Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia The Score IRT process – no single raw score applicable to all forms Slight variations to maintain consistent overall difficulty – anchor exam

5. Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia The score, continued, unfortunately So what everyone gets is the equated score, right? If only it were that easy…

6. Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia The score, continued, unfortunately The Scaled Score Zone, or Ketchup vs. Catsup— the debate rages on “70” versus “75” Where does it exist? Why does it exist? How many English majors does it take to develop an apt simile? 7075

7. Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia Now that the basics are covered… Who gets what right away: Passing Candidates

8. Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia Now that the basics are covered… Who gets what right away: Failing Candidates

9. Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia Now that the basics are covered… Who gets what right away: Regulatory Boards

10. Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia Summary reports Every year, to boards

11. Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia Choose your cliché The greatest thing since sliced bread….or something else entirely? Our great idea: reports to schools

12. Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia School data What the schools get

13. Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia How could anything go wrong? Misuse – program outcome, marketing Requests for more data Fingerpointing

14. Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia What does this have to do with my topic? Reporting service highlighted relationship between regulatory boards and schools Some boards interested in school data

15. Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia Data salad, hold the context Requests and questions from ASWB member boards Subscores? Jurisdictional pass rates? School data shared with everyone?

16. Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia Data salad, hold the context ASWB’s initial answers: No

17. Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia What’s our problem anyway? The three Ps: philosophy, psychometrics, potential litigation

18. Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia What’s our problem anyway? Philosophy Whose data is it?

19. Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia What’s our problem anyway? Psychometrics 150-item test with content areas ranging from 3 – 22 percent of the examination. Schools—and sometimes, entire jurisdictions—with small testing populations, even smaller first-timer numbers.

20. Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia What’s our problem anyway? Potential litigation Misuse/misrepresentation of data Hiring/firing/program changes based on poor data Would we prevail? Probably. But at what cost?

21. Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia Coming soon to an association near you Protection of the exam meets service to members

22. Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia Under consideration… Compilation of jurisdictional pass rates, with identification where allowed All school data reported to boards (in addition to existing program)

23. Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia What did we learn? A victim of our own success? Licensing (and the exam)more important than ever Fewer exemptions More importance among employers Recognition by educational programs

24. Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia What did we learn? Looking for outcomes in all the wrong places Test data can get caught up in political and economic pressures faced by schools

25. Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia What did we learn? Member education never stops Turnover not just limited to board members Multidisciplinary boards increase the need for education (“If they can do it why can’t you?”) Culture of profession regulated impacts expectations, available options

26. Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia What do we still need to learn? Information culture – has the data train left the station? Expectations “Good” data vs. “transparency” Whose data is it anyway?

27. Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia What do we still need to learn? Association culture – when is service a disservice? Long range planning, goal identification: don’t forget overall profile Tough decisions: sometimes you have to make them

28. Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia And you thought I’d never shut up Troy Elliott, Communications Director Association of Social Work Boards 400 South Ridge Parkway, Suite B Culpeper, VA 22701 800-225-6880 Email: info@aswb.org www.aswb.org