Student Opinion Survey It’s required…how do we make meaning out of it? How can results be the start of action?

This is radical: We need to decide in advance what we want from the SOS, not just be unhappy (or happy) with what we get

A possible first step Find a partner, a functional office like Student Life or Academic Services

How in the heck do we do that? Identify areas of concern Look at other assessments: what makes us anxious? Look at data from past SOS administrations: what do we not want to share? What in the environment/ atmosphere/ milieu keeps us up at night?

Be strategic with local questions Look at all the opportunities for local questions In a given time frame, keep the local questions largely the same Compare from one survey to the next

Four Phases Prep by IR –Decide local questions –Decide areas of interest Decide on the meaning of the data –To what will you respond? –How will you report? Dissemination of data –Who will see the data? –When? –In what format? Take action –Who gets to say “Houston, we have a problem?” –Who gets to say “This is what we’ll do to respond?”

Prep Respondents Yes, have a campaign –In 2006, you said X, Y, Z –We did A, B, C to address –Respondents want to know that what they said made a difference –Respondents are volunteers and the best volunteers are prepared for the task

Prep Respondents In the very few areas you are specifically interested in, explain to respondents what the content of an item means Give them a rubric: what does “Very Satisfied” mean? What does “Very Dissatisfied” mean?

Now Here’s a Rubric!

Prep Ourselves To what will we pay attention? Decide in advance Keep it to a small number (yes, that means everything is not #1) Create clusters of questions around these areas, using local questions

Prep Ourselves It’s not just an IR job Needs high level of involvement To whom will you report results? These are the people/offices you need to work with before the administration Urge these folks to commit to action

Build an Appetite for the Data Who are the natural customers of SOS data? What is the best way to provide the data to them? How do we get feedback about what they did in response to the data?

Prep Ourselves Conduct focus groups with students a year before the survey administration Focus on the areas with which you are concerned Or the areas in which you were unhappy with the results

Make meaning via triangulation of the Data Results from the specific administration Comparisons of means to past administrations - trends Sector rankings Always display data in context

Understand the limitations of the instrument –The SOS is a once-over lightly survey of many items –Most content areas have one question –Broad administration in a semi-controlled environment –Points us to more research or to action where general perceptions are validated –Is neither everything nor nothing

Everything is Relative (thanks Mr. Einstein) If we are doing ‘badly’ in areas that are of top priority to us, then we need action If the results are mixed, then we need to find out more information If the area is not in our top 10, then ‘who cares’? A plan to improve would involve changes in areas (see #1), and also a metaplan

Conversation based on results of SUNY New Paltz 2006 data A few 2006 SOS positives: –We are #1 in “Contribution to understanding/appreciating ethnic/cultural differences” –We do well in helping students develop an open mind to the opinions of others –What do we do to respond to these findings?

A few New Paltz not-so-positives: –We rank #12 out of 12 in the comprehensive sector in ‘helping students get a part-time job’ –Plus, our average mean is the lowest in 4 SOS administrations –Under what conditions would we do nothing about this finding?

SUNY NP ranked #12 out of 12 in satisfaction with advising There is no clear trend in comparing means over time What should we do?

SUNY NP is #12 out of 12 in the “Purposes for which the student activity fee is used” What should we do? What was going on at the time?

Meaning & Action You can’t take action without meaning You can’t find meaning without a plan You can just administer the survey, and then be faced with a mess of responses that don’t yield much meaning The result: no action