A self study is not an assessment Assessment: the continuous process of gathering information to determine and improve progress toward specific outcomes (Aiken- Wisniewski et al., 2010 Robbins & Zarges, 2012 Troxel, 2008) Assessment
A self study is not an evaluation Evaluation: the process by which individuals, teams, or practices are judged often with the use of specific ratings (Robbins 2009, 2012) Evaluation
But... A self study may anticipate an assessment process or evaluation. “Start with the end in mind.” Aiken-Wisniewski et al., 2010
For the self study, focus on PDOs Concentrate on process/delivery outcomes (PDOs) Save student learning outcomes (SLOs) for a future assessment
A self study is... A formal report grounded in strategic and consistent consistent data-gathering An opportunity to identify common practice among disparate or remote professionals A non-evaluative “snapshot” of professional practice
Why might a self study be useful? To educate administrators To appeal for funds or resources To connect practice to vision/mission To reveal practice in a satellite or hybrid advising model Center for Academic Success/O UE Colle ge of Scien ces Biolo gy Che mistr y Physi cs MathEAS Colle ge of Engin eerin g CEE ME/ NRE CHBEAEISyE Scheller College of Business Administ ration Ivan Allen Colle ge of Liber al Arts ECO N INTAHTSLMCPUBPALIS College of Computi ng CMCS College of Architec ture ARCHID
Decentralized Advising at Georgia Tech Center for Academic Success/OUE College of Sciences Biology Chemistry, Biochemistry Physics, Applied Physics Math, Applied Math EAS College of Engineering CEEME/NRECHBEAEISyEECE Scheller College of Business Administration Ivan Allen College of Liberal Arts ECONINTAHTSLMCPUBP ALIS, GEML, IAML College of Computing CMCS College of Architecture ARCHID
Reflection 1 What questions might a self study answer for your home institution?
Where would I start? Establish objectives for your study: – To gather data about different advising practices – To learn more about the student experience – To discover resource inequities in different advising units – To identify best practices
Whom might I consult on campus? Institutional Research Office of Assessment Larger on-campus advising network Any unit that has completed a self study
Whom might I consult off campus? NACADA – Clearinghouse – Assessment Institute Webinars – Innovative Educators – Academic Impressions Professionals at other schools Completed self studies
Tip from experience! Make the study a team effort – Identify stakeholders – Cultivate buy-in – Share the labor – Make each other accountable Get consistent help from students or staff – Transcription and data entry
Reflection 2 Identify two campus partners you might consult to help plan the self study. What stakeholders might participate in your self study team?
Execution Gather statistical information through surveys Gather detailed and anecdotal information through interviews Organize and categorize Summarize parts and whole
Execution: Survey Vet your survey with assessment or institutional research team Test the survey on actual advisors Don’t reinvent the wheel: use automated survey generators like Survey Monkey, SurveyGizmo, Qualtrics
Tip from Experience! Consider the best audience for different types of questions Consider separate surveys – Comprehensive to capture everybody’s experience – Selective for authoritative information
Reflection 3 Will your survey be selective or comprehensive? Will you need more than one survey for multiple audiences?
Execution: Interview Schedule appointments early Send an email with an overview or agenda and a request for any specific materials Meet advisors in their work space Take pictures Stick to a standardized script, ask follow up questions as warranted
Tips from Experience! Consider starting with face-to-face interviews and then moving on to statistical data Summarize interview findings and follow up immediately Look for “low-hanging fruit” that can be addressed with immediate action
Reflection 4 How might you select and categorize information for a face-to-face interview? Student ExperienceAdvisor Experience OrientationHiring RegistrationTraining At-risk performanceAssessment Processes Change of Academic StandingProfessional Development
Execution: Organize Information Keep materials organized as you go Enter hand-written information into spreadsheet or database
Delivery Draft the report – Consider the audience – Keep it lean Revise the report – See it again with fresh (and different!) eyes Include visual aids and personal testimony Present information intentionally Include appendices with details
Promotion Present information to different stakeholders Spread the word Anticipate next steps AudienceFormat Provost, Dean, VPReport, executive summary AdvisorsReport, town hall Institute ResearchReport, meeting StudentsReport, meeting
Reflection 5 For the stakeholders you identified in reflection 2, determine a format and occasion for presenting the study.
References Aiken-Wisneiwski, S. (Ed)., (2010). Guide to Assessment in Academic Advising (second edition). [Monograph No. 23]. Manhattan, KS: The National Academic Advising Association. Robbins, R. (2009). Evaluation and assessment of career advising. In Hughey, K. Burton Nelson, D., Damminger, J. and McCalla-Wriggins, B., (Eds) Handbook of Career Advising (chapter 12). San Francisco: Jossey-Bass. Robbins, R. & Zarges, K.M. (2011). Assessment of Academic Advising: A Summary of the Process. Retrieved from NACADA Clearinghouse of Academic Advising Resources Web Site: www.nacada.ksu.www.nacada.ksu Troxel, W.G. (2008). Assessing the effectiveness of the advising program. In V.N. Gordon, W.R. Habley, and T.J. Grites, Academic Advising: A Comprehensive Handbook (2 nd edition) (pp. 286-295). San Francisco: Jossey-Bass.
Contact Me! Shannon Dobranski Director, Center for Academic Success Georgia Tech firstname.lastname@example.org