Intern Incorporation Into An Interview and Selection Process Victor S. Wolski, PhD, Shelley M. Prisco, PsyD, Christina Wilder, PsyD, and Nile Wagley, MS Northwest Georgia Consortium (NGC) Internship Rome, GA Benefits of Incorporating Interns Preliminary Intern Application Review Interview Questions Interview Format Each group of 2-3 faculty or current interns has a question to ask. The questions are determined by the Director and the Associate Director with input from everyone. The following is one example. Topic: Clinical Work Conceptualization Question: Please select and describe a piece of your clinical work as you would for a supervision session. Poor response: The theoretical framework is either not stated or cannot be identified or linked to the work presented. The ability and willingness to engage in dialogue is limited by resistance or lack of conceptual understanding. Average response: Identifies or clarifies adequately the theoretical framework used to conceptualize the approach taken. Engages in dialogue that demonstrates receptivity and use of new information in analysis or consideration of alternatives. Points to some clinical literature and/or empirically supported treatment literature as basis for the approach used. Excellent response: Does all of the above with a degree of thoroughness and discrimination more characteristic of an advanced intern. Demonstrates the use of the questions and discussion to generate new learning on the topic and/or develop awareness or insight into themselves personally. Teaches interns about various administrative and organizational responsibilities in staff selection in a supportive environment Provides a different perspective during group discussions on ranking Potentially increases applicants comfort during interviews Shows applicants the collaborative approach emphasized in the internship experience at NWGC Allows interns an opportunity to evaluate interview strategies for their own postdoctoral and employment interviews Educates interns on legal and ethical issues around staff selection and hiring Interview Days begin with the Director and current interns welcoming the applicants. Current interns then give a 1-hour PowerPoint presentation on the internship, including pros and cons of the internship, and answer questions. Current interns bring the applicants to a 15-minute “meet and greet” with all the internship faculty and refreshments (provided by faculty). Current interns show each applicant to an office where he or she will stay for the individual interviews with faculty and interns rotating. Applicants are interviewed individually for 30 minutes by a group of 2-3 faculty or the 3 current interns. The 5 groups rotate around to each applicant such that each applicant is interviewed for a total of 2.5 hours. After the interviews, everyone returns to the break room. The Director answers last questions and has closing remarks *** Please do not mark in the applicant file; the interns have access to them once hired. If you would like to flag a part of the application, use a sticky note. Applicant Name: Applicant Program: APA Program? Yes No School_______________________ Reviewer Name: When reviewing this application I found: Please put a Y or N or # Circle One Transcript Concerns Any Cs, Ds or Fs? Any Incompletes? ____ Yes No Grammatical/Punctuation Errors Writing Sample Concerns Is the report integrated? Does the evaluation support the diagnosis? Applicant lacks Adequate Experience Base How many Adult therapy clients? How many C&A therapy clients? How many Adult evaluations? How many C&A evaluations? How many Rorschachs using Exner Objective and Projective tests? Inpatient Experience? Outpatient Experience? Concerns with Letters of Recommendations Potential problems with “Goodness of Fit” Work with our faculty? Work with Treatment teams/organizations? Want to work with this population? Does the person seem to want generalist training? _____ Other: (specify) Current Interns and the Process Current NWGC interns participate in every step of the process: Reading applications Reviewing the website and the brochure for accuracy Completing data entry sheets on applications, including an overall global rating Narrowing the applicant pool down to 25 for interviews through group discussion and review of the data entry sheets Preparing and presenting a presentation to applicants in a group setting at the beginning of each interview day 6. Interviewing individual applicants on a pre-determined question, as well as spontaneous questions 7. Providing feedback on applicants after each interview day, including rank ordering applicants 8. Participating in a final group session to determining submitted rankings after all interview days are complete 9. Answering questions from applicants via email and phone 10. Evaluating the interview and selection process, and suggesting changes as needed. Focus of Interviews 1.     Adequacy of academic training and clinical familiarity with populations served by the internship 2.     Competencies determined by application and interview questions 3.     Focus on assessment of training needs and our ability to meet those needs 4.     Related life experiences adequate and relevant to both organizational characteristics (ie. Southeastern U.S., urban, suburban, growth rural populations) and generalist practitioner focus at the internship (autobiographical statement extremely valuable) 5. Future career goals show commitment to public sector or not for-profit organizational setting The applicant appears to be a match for the consortium based on: Poor Average Excellent Interest in and preparation for a generalist training Interest in seriously mentally ill population Level of personal maturity relative to their training (to handle the stress and fully utilize the training experience) Flexibility and adaptability to organizations Good personality match for the training program/Clinical Training Committee Overall match for the program Northwest Georgia Consortium Internship Interviews December 18, 2008 12:30-1:00 Meet intern candidates Building 103, Conference Room A for Welcome and Introduction from current interns Diane’s Office #2-07 Intern Applicant #1 Sam’s Office #2-76 Intern Applicant #2 Julie’s Office #2-77 Intern Applicant #3 Intern’s Office #2-82 Intern Applicant #4 Shelley’s Office #2-95 Intern Applicant #5 1:00 – 1:15 Forensic Services Conference Room as “Break room” General Welcome from Director of Training, and introductions from all psychologists Refreshments 1:15 – 1:45 Dr. Diane Vendryes Dr. Tonita Baines Dr. Julie Oliver Dr. Richard Stewart Dr. Sam Perri Dr. Ryan Beck Dr. Victor Wolski Dr. Shelley Prisco Rhea Holler, MA Lindsay Klieman, MA Nile Wagley, MS Dr. Tim Chomyn Dr. Arlene DeRienzo 1:50 – 2:20 2:25 – 2:55 3.00 – 3:30 Nile Wagley, MS 3:35 – 4.05 4:05 – 4:20 Please return to the Forensic Conference Room, Break room, for a wrap-up meeting. This is an opportunity to ask any additional questions and review the selection/ranking process of MATCH. You will also be provided a listing of all phone numbers and email addresses for contacting any of the supervising psychologists not present today. Approximate Timeline Relevant Legal Issues Based on federal laws, such as the Civil Rights Act of 1964, the Age Discrimination in Employment Act of 1967, and the Americans with Disabilities Act of 1990, interns are instructed not to ask about an applicant’s: -Age -Marital or family status -Sexual orientation -Religious affiliation -Physical condition or limitations As a general rule, interns are instructed to ask applicants questions only directly relevant to the applicant's qualifications or to the internship position and duties.  When in doubt, don't ask! August 30th – Update website and APPIC September/October – Receive applications November 1st – Begin reviewing applications; at least 2 faculty/interns read and comment on each application November 14th – Due date for applications December 4th – Director and Associate Director review all applications and reviews, and narrow the pool to 25 for interviews Five Thursday afternoons: December 18th to January 22th- Interview 5 applicants and then meet as a group to rank order/discuss for 30 minutes January 29th – Final ranking meeting February 4th – Submit Rankings February 20th – MATCH Day; Director contacts interns February 28th – Email all applicants zoomerang.com survey March 15th – Clinical Training Committee and interns review survey feedback and determine what, if any, changes will be made to next year’s process Relevant Ethical Issues Potential ethical issues when taking on the task of the recruitment and the selection of staff or future interns include: Avoiding harm Maintaining integrity and accurate representation of the program/avoiding false or deceptive statements Preventing unfair discrimination Recognizing and disclosing conflicts of interest that impairs objectivity Refraining from requiring disclosure of personal information regarding sexual history, psychological treatment, and relationships with parents, peers, and spouses

Intern Incorporation Into An Interview and Selection Process Victor S. Wolski, PhD, Shelley M. Prisco, PsyD, Christina Wilder, PsyD, and Nile Wagley, MS Northwest Georgia Consortium (NGC) Internship Rome, GA Weekly Rankings Data MATCH Post Interview Applicant Survey – Part 2 Post MATCH Survey Utility Director calls to confirm MATCH and congratulate the future intern Director facilitates exchanging names, phone numbers and email addresses of other interns Director establish expectation of meeting Psychology group, faculty and interns, at “Hello and Good-bye” party last Saturday in August for transition into internship and the social/professional group MATCHes consistently within top 50% of rankings Intern Applicant Rankings Appl Psy Int Total Aver Final Rank 12/18 A1 4 5 3 2 46 3.54 15 A2 1 23 1.77 A3 41 3.15 10 A4 32 2.46 9 A5 53 4.08 17 1/8 B1 n/a 18 1.64 B2 44 B3 47 4.27 16 B4 B5 38 3.45 20 1/15 C1 25 2.08 7 C2 56 4.67 21 C3 3.17 13 C4 29 2.42 11 C5 31 2.82 19 1/22 D1 26 2.36 14 D2 24 2.18 8 D3 3.73 22 D4 1.73 6 1/29 E1 1.55 E2 1.91 E3 33 3.00 12 Was there any information not provided on the website or in the brochure that would have been useful? What did you like most about the NWGC interview process? What did you like least about the NWGC interview process? What do you consider to be the strengths of the NWGC? What do you consider to be the weaknesses of the NWGC? What factors made the NWGC drop in your ranking compared to other sites you visited and ranked? What other feedback can you provide for us? One use of survey results is to evaluate the consistency of applicants’ perceptions of various aspects of the NWGC, including whether applicants agree or disagree with the statement “overall a positive experience.” Post MATCH Surveys A 15-item survey has been distributed via email and Zoomerang.com to applicants after their interviews since 2005. 7 of the 15 items are on a 1-5 Likert Scale (e.g, the NGC website provided clear and accurate information with 1 being strongly disagree and 5 being strongly agree) 8 of the 15 items are open-ended (e.g., What did you like most about the NWGC interviews? What do you consider to be the weaknesses of the NWGC? Response rate over the past 5 years has ranged from 36% (2008) to 65% (2009). Total number of surveys collected over the past 5 years is 58 Themes from Post MATCH Surveys Three overarching themes can be qualitatively derived from the open ended questions on the surveys. 1. Interview Logistics Likes: Meeting all the training staff and interns, receiving an overview of the internship from interns, and staying in one room while the interviewers rotate Dislikes: Having brief individual interviews, being interviewed by multiple people at one time, and not having a tour of the facility 2. Rotation Specifics Likes: Forensic opportunities, assessment opportunities, and diversity in rotations Dislikes: Not knowing the theoretical orientation of training staff, perceived lack of therapy opportunities relative to assessment, training in the Rorschach 3. Internship Characteristics Likes: Relaxed/collegial atmosphere, organized internship with clear and updated materials Dislikes: Traveling over an hour to a rotation, amount of stipend, and perceived lack of diversity in faculty and clientele Final Ranking Meeting Usually one hour with all available faculty and interns Interviewed pool of 25 sorted into “excellent,” “very good” and “good” categories based on initial rankings after interviews Anyone may propose an “elimination” or non-ranking of any applicant at any time in the meeting A suggested “elimination” from ranking can be negated by an expressed commitment by a faculty member to work with the individual, and the Director and Associate Director concur that such a commitment would most likely result in successful completion of the internship. After applicants are placed into categories, they are assigned rankings starting with those in the “excellent” category working through all three groupings with slight movement between groupings Another use of survey results is to determine where changes could be made to improve the process and the internship. For example, after looking at the data on whether applicants agreed or disagreed with “I felt the individual interview sessions allowed faculty to get to know me,” it was decided to reduce the scope and number of questions discussed to one question/topic per individual interview segments in 2009. Post Interview Applicant Survey – Part 1 Total distributed = 23 N=15 1 - Strongly Disagree, 2 - Disagree, 3 - Neutral, 4 - Agree, 5 -Strongly Agree Average Score The NWGC website provided clear and accurate information about the consortium. 4.2 The NWGC brochure provided clear and accurate information about the consortium. The introductory group meeting on interview day was helpful in orienting me to the consortium. 4.4 The introductory group meeting helped me to feel more comfortable with the interview process. 4.3 The individual interview sessions helped answer my questions about the Consortium. 4.5 I felt the individual interview sessions allowed faculty to get to know me. The overall interview process was a positive experience. Ethical Dilemmas Ethical dilemmas are discussed with interns throughout recruitment and selection. Ethical dilemmas encountered by interns primarily center around conflict of interest/objectivity and include: - Sharing an educational or work background with an applicant Knowing the applicant personally through school or work Knowing the applicant as a result of internship interviews the year before Feeling uncomfortable giving low ratings or pointing out weakness of an applicant Having to remind an applicant of the intern’s involvement in recruitment and selection because the applicant is too casual or disrespectful during an interview or in an email It is interesting to note that several factors mentioned in our post MATCH surveys, such as up-to-date materials, time with faculty and interns, theoretical orientation of staff, and geographical location/travel time, are consistent with previous literature on intern applicants' decision-making processes and variables important to students in interviewing and selecting internship sites (Stedman, 2006, Vowles and McNeil, 2000).