1. Using Technology to Efficiently and Effectively Gather Information from Recent Alumni and Employers April 2009 Florida Association of Community Colleges Institutional Effectiveness, Planning, & Professional Development Commission 2009 Joint Annual Spring Conference

2. 2009 Joint Annual Spring Conference April 2009 April 16, 2009 Florida Association of Community Colleges2 Alumni and Employer Surveys Presenters  Nicole Carr, Assessment Coordinator for Academic Programs  Maggie Tymms, Assessment Coordinator for Academic Programs  Dr. James Coraggio, Director of Institutional Research and Effectiveness

3. 2009 Joint Annual Spring Conference April 2009 April 16, 2009 Florida Association of Community Colleges3 SPC Background  SPC, established in 1927, is the oldest 2-year college in Florida  First Community College in Florida to offer 4 year degrees (2002)  9 Campuses throughout the county  2007-08 FTE: 16,086 (LD), 1,415 (UD)  Opening Fall 2008 credited enrollment: 27,076  Annual 2007-08 headcount: 62,200

4. 2009 Joint Annual Spring Conference April 2009 April 16, 2009 Florida Association of Community Colleges4 Institutional Effectiveness “Institutional Effectiveness is the integrated, systematic, explicit, and documented processes of measuring performance against the SPC mission for purposes of continuous improvement of academic programs, administrative services, and educational services offered by the college.” “Closing the Loop”

5. 2009 Joint Annual Spring Conference April 2009 April 16, 2009 Florida Association of Community Colleges5 Performance Improvement  From Compliance to Performance Improvement

6. 2009 Joint Annual Spring Conference April 2009 April 16, 2009 Florida Association of Community Colleges6 Agenda  Student Assessment Points  Purpose of Follow-up Surveys  Alumni/Employer Survey Process  Dissemination of Survey Results and Reports

7. 2009 Joint Annual Spring Conference April 2009 April 16, 2009 Florida Association of Community Colleges7 Student Assessment Points Students surveyed multiple times

8. 2009 Joint Annual Spring Conference April 2009 April 16, 2009 Florida Association of Community Colleges8 Purpose of Follow-up Surveys Recent Alumni Survey was designed to:  Gain insight into alumni’s perceptions of the quality of general education and career preparation  Determine the current activities of former students such as employment information and educational status

9. 2009 Joint Annual Spring Conference April 2009 April 16, 2009 Florida Association of Community Colleges9 Purpose of Follow-up Surveys Employer Survey was designed to:  Gain insight into an employer’s perceptions of hired SPC graduates in the areas of key general education competencies and foundational skills  Provide the employee’s major job responsibilities and wage information, as well as employer’s willingness to hire another SPC graduate  Identify employer’s willingness to support various college activities

10. 2009 Joint Annual Spring Conference April 2009 April 16, 2009 Florida Association of Community Colleges10 Purpose of Follow-up Surveys State Requirements  Survey data are also collected for state required annual Placement & Follow-up Report (FETPIP: Florida Education Training Placement Information Program) and supplemental information to meet workforce criteria

11. 2009 Joint Annual Spring Conference April 2009 April 16, 2009 Florida Association of Community Colleges11 Alumni/Employer Survey Process Previous model  Labor intensive  Manually tabulated paper surveys  Telephone interviews conducted using overtime hours

12. 2009 Joint Annual Spring Conference April 2009 April 16, 2009 Florida Association of Community Colleges12 Alumni/Employer Survey Process Issues with previous model  Costly survey process  Declining college budget  Increasing demand for accountability  Outdated contact information  Students over-surveyed  Declining response rates  Conducted in isolation

13. 2009 Joint Annual Spring Conference April 2009 April 16, 2009 Florida Association of Community Colleges13 Alumni/Employer Survey Process Goals of new survey process A.Improve data collection process B.Enhance quality of survey forms C.Employ Industry Best Practices D.Include discipline specific information E.Increase response rates

14. 2009 Joint Annual Spring Conference April 2009 April 16, 2009 Florida Association of Community Colleges14 A. Improve Data Collection Process Data Collection Process  Eliminated telephone calls  Automated the process (email merge)  Employ online survey submission (Zoomerang TM )  Obtain accurate email addresses  Use multiple points of contact

15. 2009 Joint Annual Spring Conference April 2009 April 16, 2009 Florida Association of Community Colleges15 A. Improve Data Collection Process Survey Schedule  Recent Alumni Survey  Graduates surveyed six months after the end of session (Three times per year)  Employer Survey  Employer of graduates surveyed at the end of academic year (Once per year)

16. 2009 Joint Annual Spring Conference April 2009 April 16, 2009 Florida Association of Community Colleges16 A. Improve Data Collection Process First email Second emailThird email First mailing Second mailing 2 weeks Recent Alumni Survey Multiple Points of Contact 2 weeks

17. 2009 Joint Annual Spring Conference April 2009 April 16, 2009 Florida Association of Community Colleges17 B. Enhance Quality Of Survey Forms Improved Survey Forms  Better “look and feel” of survey format  Developed scannable paper forms  Use online branching models to reduce survey length  Design items to gather only pertinent data

18. 2009 Joint Annual Spring Conference April 2009 April 16, 2009 Florida Association of Community Colleges18 C. Employ Industry Best Practices Don Dillman’s Five Elements  Respondent- friendly questionnaire  Up to five contacts with the respondent  Inclusion of stamped envelope  Personalized correspondence  A token financial incentive that is sent with the survey request

19. 2009 Joint Annual Spring Conference April 2009 April 16, 2009 Florida Association of Community Colleges19 Online Survey: Email and Form

20. 2009 Joint Annual Spring Conference April 2009 April 16, 2009 Florida Association of Community Colleges20 Paper Survey: Letter and Form

21. 2009 Joint Annual Spring Conference April 2009 April 16, 2009 Florida Association of Community Colleges21 C. Include Program Specific Information Program specific information  Meet with Deans/Program Directors  Discuss requirements for program accreditation and timelines  Include their necessary program specific items

22. 2009 Joint Annual Spring Conference April 2009 April 16, 2009 Florida Association of Community Colleges22 D. Increase Response Rates Create awareness and convey importance  Involve program faculty in the process  Request faculty assistance to inform students  Include a flyer with graduation materials  Coordinate with programs that hand- carry surveys to local employers

23. 2009 Joint Annual Spring Conference April 2009 April 16, 2009 Florida Association of Community Colleges23 Alumni Survey Response Rates  Summer 07

24. 2009 Joint Annual Spring Conference April 2009 April 16, 2009 Florida Association of Community Colleges24 Alumni Survey Response Rates  Fall 07

25. 2009 Joint Annual Spring Conference April 2009 April 16, 2009 Florida Association of Community Colleges25 Alumni Survey Response Rates  Spring 08

26. 2009 Joint Annual Spring Conference April 2009 April 16, 2009 Florida Association of Community Colleges26 Alumni Survey Response Rates *Summer 2008 in progress

27. 2009 Joint Annual Spring Conference April 2009 April 16, 2009 Florida Association of Community Colleges27 Alumni Survey Response Rates Source of Responses 2007-08

28. 2009 Joint Annual Spring Conference April 2009 April 16, 2009 Florida Association of Community Colleges28 Alumni Survey Response Rates

29. 2009 Joint Annual Spring Conference April 2009 April 16, 2009 Florida Association of Community Colleges29 Alumni Survey Response Rates Overall Response Rate *Summer 2008 in progress

30. 2009 Joint Annual Spring Conference April 2009 April 16, 2009 Florida Association of Community Colleges30 Alumni Survey Response Rates Graduate Year Surveyed Number Surveyed Number Responded Percent Responded Number of Online Responses Percent of Responses Online 2007-083,7011,01326.5%52251.5% 2006-073,95365816.7%30045.5% 2005-063,75698226.1%23123.5% 2004-053,4171,49443.7%1006.6%

31. 2009 Joint Annual Spring Conference April 2009 April 16, 2009 Florida Association of Community Colleges31 Response Rates - Employer Other Responses Recent Alumni Surveys Sent Employer Surveys Sent Is a 44% Employer Response Rate Good? COE Responses With Permission Responses Without Permission

32. 2009 Joint Annual Spring Conference April 2009 April 16, 2009 Florida Association of Community Colleges32 Disseminating Information  College wide access to reports  http://www.spcollege.edu/Central/Surveys/SurveyList.htm http://www.spcollege.edu/Central/Surveys/SurveyList.htm  Program Specific Reports  Published on the Educational Outcomes website  http://it.spcollege.edu/edoutcomes

33. 2009 Joint Annual Spring Conference April 2009 April 16, 2009 Florida Association of Community Colleges33 Disseminating Information

34. 2009 Joint Annual Spring Conference April 2009 April 16, 2009 Florida Association of Community Colleges34 Disseminating Information

35. 2009 Joint Annual Spring Conference April 2009 April 16, 2009 Florida Association of Community Colleges35 Disseminating Information

36. 2009 Joint Annual Spring Conference April 2009 April 16, 2009 Florida Association of Community Colleges36 Program Specific Reports

37. 2009 Joint Annual Spring Conference April 2009 April 16, 2009 Florida Association of Community Colleges37 Lessons Learned  Do not expect that students will remember their ID#s  Administer the survey soon after the end of the session (six months)  Use multiple points of contact (two weeks)  Find the most recent contact information  Partner with programs to acquire up-to-date information

38. 2009 Joint Annual Spring Conference April 2009 April 16, 2009 Florida Association of Community Colleges38 Lessons Learned  Make students aware of the importance of the survey process (survey announcements)  Provide contact updates to college data systems (PeopleSoft, Alumni Foundation)  Monitor results and make changes as necessary  More accurate data (quicker turn around of results) with automated processes

39. 2009 Joint Annual Spring Conference April 2009 April 16, 2009 Florida Association of Community Colleges39 Future Direction  Ultimate goal is to provide Provosts (Deans) and Program Directors ‘timely’, ‘relevant’ ‘accurate’, and ‘interpretable’ information regarding their programs and their performance outcomes  Multi-measure ‘Dashboard’ style reports (Business Intelligence/Performance Measurement)

40. 2009 Joint Annual Spring Conference April 2009 April 16, 2009 Florida Association of Community Colleges40 Questions/Next Steps

41. Using Technology to Efficiently and Effectively Gather Information from Recent Alumni and Employers April 2009 Florida Association of Community Colleges Institutional Effectiveness, Planning, & Professional Development Commission 2009 Joint Annual Spring Conference