1. Alexa Lamm, Ph.D. Diane Craig University of Florida American Evaluation Association October 25, 2012 USING EVALUATION TO BUILD A FUTURE FOR EXTENSION

2. 1.Discuss the UF/IFAS Extension strategic planning process 2.Review three ways evaluators played an integral role in the Extension strategic planning process at the University of Florida  Delphi  Listening Sessions  Return on Investment materials 3.Discuss advantages and disadvantages of each approach 4.Cover next steps in the implementation of an Extension strategic plan 5.Lead discussion surrounding the evaluators role in building a future for Extension SESSION OBJECTIVES

3. UF/IFAS EXTENSION STRATEGIC PLANNING Guiding principles Strategic Goals Actions Outcomes Vision/Mission

4. UF/IFAS EXTENSION STRATEGIC PLANNING Data collection Delphi Listening Sessions Summit Develop Objective statement Strategy development Finalization Creation of ROI materials Presentation to RACs, BOT

5. EVALUATING STAKEHOLDER NEEDS IN THE EXTENSION STRATEGIC PLANNING PROCESS: A DELPHI APPROACH  Identified 41 experts for the panel  Three rounds 1.Open-ended question  34 responded 2.Responses were combined & panel was asked to rate their level of agreement (1 – 5)  36 responded 3.Responses receiving 3.5+ sent out to reach consensus (75%+ agreement)  36 responded

6. Objective #1 Identify the fundamentals essential to achieving the overarching goals of the University of Florida and identify areas in IFAS for investment, in light of its current position, the research environment, and social and academic considerations. Identify knowledge gaps & use to drive new initiatives Conduct a formal, periodic needs assessment with stakeholder groups Effectively deliver IFAS information to rural and urban clientele Improve communication technology & public relations programs Develop academic programs that prepare graduates for current workforce Develop courses that include applied skills & internships Encourage faculty to include global perspectives in their teaching Form multi-disciplinary teams between faculty and stakeholders Reorganize distance education to meet student & faculty needs

7. Objective #2 Develop solutions for issues that affect agricultural, human, natural resources, and life sciences. Water Address water quality issues Develop/educate others on practical water quality standards (BMPs) Regulation Educate legislature on the benefits of agriculture and impacts of regulations Develop programs to help producers understand and implement government regulations Promote management of emerging pests & exotic diseases Citrus greening cure Focus research on minimizing, mitigating or avoiding environmental impacts Communicate successes to stakeholders including leaders, activist groups, and media

8. Objective #3 Hire and retain the best people in their fields, including providing them with a productive environment, tools and resources. Survey best practices among units, develop trainings, and spread them to other units for management efficiency Eliminate redundancies in reporting Encourage and reward interdisciplinary collaboration, research, and academic programs Target key issues Maximize allocation of faculty/financial support Communicate results to public and policy makers through educational programs & materials Flatten IFAS administrative structure Enforce shared governance and ensure consistency with business and operational structures and policies

9. Objective #4 Create a more sustainable approach to agriculture while recognizing wider important social and economic parameters. Promote sustainable agricultural programs that support land stewardship and food production Address Florida environmental issues by developing sustainable practices within Florida’s diverse population Incorporate sustainability measures into BMP’s made by IFAS Locate funding for investigating sustainable agricultural strategies and simplify grant acquisitions for faculty Develop tools that producers and Extension can use to educate others on the importance of domestic agriculture and environmental impacts

10. Objective #5 Develop organizational efficiencies and effectiveness to promote and acquire adequate resources to fulfill the mission. Reorganize Extension Service to make it more sustainable financially while providing maximum support to customers Analyze business practices, organizational structures, and communication efforts to increase output Improve transparency in reporting impact/ROI for IFAS funding & request supported industries to match funds with IFAS Eliminate low priority, low performance programs Educate the public and policy makers about IFAS’s expertise in RTE, human resources, and life sciences Integrate consistent message into outreach, recruitment and youth development efforts Embrace growth opportunities for food production technologies within IFAS

11. Feedback was received from:  Three Deans  Regional Advisory Committees (5)  Administrative Council – Department Chairs and Center Directors  College of Agriculture Faculty Council  President and Board of Trustees Results were used to establish benchmark areas within each of the IFAS objectives Extension aligned with their objectives and outcomes DELPHI RESULTS WERE SHARED

12. Advantages  Online administration limited travel to gain input from stakeholders  Broad perspectives narrowed down to specific objectives  Succinct representation from all stakeholder groups  Quantitative data made available to drive decision making Disadvantages  Lack of understanding of the Delphi process  Very scientific compared to previously used listening sessions  Lack of direct contact with respondents  Limited number of participants  Misplaced judgment on results being biased DELPHI ADVANTAGES & DISADVANTAGES

13. Listening Sessions  Issues and trends facing communities over the next decade and how can Extension help?  Statewide, 67 counties (Feb-May 2011)  10 regional meetings to discuss findings with Extension administrators and state specialists (May-June 2011)  Campus departments and Research and Education Centers (REC) review county reports and conduct listening sessions with advisory committees (June-August 2011) FROM EVALUATION TO ACTION: BUILDING STRATEGIC AND AGILE ISSUE-BASED EXTENSION TEAMS

15.  Goal: To develop a clear path of action to support the state’s economy, environment, and people in a way that is relevant and flexible to meet the changing needs of Florida’s communities.  Held seven months later – March 2012  1.5 days on University of Florida campus  Remarks from deans, vice president  Small groups to discuss the key statewide issues identified during the long-range planning process  Social hour, networking opportunities EXTENSION SUMMIT

16.  300 participants (55% of pop.)  County faculty  State specialists  Observers/helpers  Administrators, incl. program leaders  Staff  Visitors  All program areas represented  Preparation – draft long-range plan sent out prior to meeting SUMMIT PARTICIPANTS

17.  Faculty and specialists with and without Extension appointments  Some were VERY familiar with initiatives while others were not even from within IFAS  Role was to lead discussion guided by clear questions developed by the Extension leadership team  At the end of each day, the facilitator presented information gathered from the group SUMMIT FACILITATORS

18.  During registration process, faculty identified their top two choices from list of initiatives and “super issues”  Required to submit a short super-issue abstract (less than 100 words) that described how the super-issue chosen as their first priority relates to the needs in your county (or the state), and how they see their programs contributing.  Attendees assigned to group based on first or second choice and their abstract. SUMMIT GROUP ASSIGNMENT

19.  Maximize organizational efficiency and effectiveness  Invest in Extension’s human capital  Increase the visibility and demonstrated impact of Florida Extension ORGANIZATIONAL GOALS

20.  SWOT Analysis (Strengths, Weaknesses, Opportunities, Threats)  What actions should be taken to achieve the goal?  Who in Extension (agents and specialists) should be involved? BREAKOUT GROUPS: GOALS

21.  Increasing the Sustainability, Profitability, and Competitive- ness of Agricultural and Horticultural Enterprises  Enhancing and Protecting Water Quality, Quantity, and Supply  Enhancing and Conserving Florida's Natural Resources and Environmental Quality  Producing and Conserving Traditional and Alternative Forms of Energy  Empowering Individuals and Families to Build Healthy Lives and Achieve Social and Economic Success  Strengthening Urban and Rural Community Resources and Economic Development  Preparing Youth to be Responsible Citizens and Productive Members of the Workforce 7 “HIGH PRIORITY” INITIATIVES

22.  List of key words and phrases provided for each initiative  Each breakout group tasked with developing a strategic action plan for the top two or three programs  For each priority program, smaller groups identify:  Target audience and their needs  Key programmatic objectives  Strategies for county and faculty to work together  Professional development needs for faculty  Measures of success BREAKOUT GROUPS: INITIATIVES

23.  Awareness and Appreciation of Food Systems and Environment  Sustainability and Conservation of Resources in Florida Communities  Financial Management for Individuals, Business Enterprises, and Communities  Opportunity for Youth to Explore STEM  Opportunity for Floridians to Embrace Healthy Lifestyles 5 “CROSS CUTTING” SUPER ISSUES

24.  Key priorities and outcomes?  What current programs fall under this super issue?  What gaps in programming currently exist?  What new initiatives or programs should be considered?  What in-service training is needed?  How to best organize current resources?  What new resources are needed? BREAKOUT GROUPS: SUPER ISSUES

25.  Does the priority program identified for the initiative contribute to any of the five super issues?  Which programs contribute to which super issues? INTEGRATION OF INITIATIVES AND SUPER ISSUES

26.  Like the statewide listening sessions, the Summit breakout groups = lots of qualitative data!  Prioritizing -- what really matters most  Issues and organizationally (resources) ANALYSIS

27.  Dean identified leadership teams for each initiative (10-25 faculty)  Each initiative was further divided into 2-5 priorities or teams.  Most initiatives have three work groups.  Initiative 5 – FCS example:  Food safety and nutrition  Housing  Family financial management  Aging well  Human development and family relationships LEADERSHIP TEAMS & PRIORITY WORK GROUPS

28.  Teams consist of county and state faculty; new is more direct involvement by academic & research unit leaders  Annual award of $5,000 for most effective team  Building capacity in marketing, communication and evaluation by identifying and training specific team members  Funded summits for each of the 7 initiatives to determine statewide programs, write plans of work for each, develop method to communicate with each other, identify future funding and collaborators  Use Social Network Analysis to benchmark teams internal and external relationships after formation and again in near future (1 yr, 2yrs, ?) AGILE AND RESPONSIVE TEAMS

29.  Priority Work Groups are identifying short, medium, and long term outcomes they plan to establish in the next five years  Outcomes are aligned with overall Extension and IFAS strategic plan  Professional development program for teams is being established to ensure short term goals are developed with accountability measures put in place  Cross-team interdisciplinary collaboration is being encouraged but at this point no accountability or reward for partnering is in place NEXT STEPS

30. Advantages  All Extension personnel focused on strategic planning at one time  Participants felt they had a voice  Buy-in from those who will be carrying out the initiatives  Focusing on what is most important – prioritize!  Thinking about larger campus, future partners Disadvantages  Demand exceeded space  Open space rather than individual rooms  Lack of professional facilitators  Lack of direction about how super issues relate to initiatives (intentional?)  Data gets lost in translation  Data overload/fatigue  Leadership of future efforts was vague SUMMIT ADVANTAGES & DISADVANTAGES

31.  Demonstrating value to legislators, governor, county managers, partners, the general public is key  Using information collected in evaluation by teams engages stakeholders  Demonstrating value to faculty by actually using their accountability data  Leverages the reporting that takes place all over campus to meet federal and state requirements RETURN ON INVESTMENT: USING EVALUATION DATA TO GAIN POLICY MAKERS’ SUPPORT FOR EXTENSION

34. ECONOMIC IMPACT  UF/IFAS Food and Resource Economics Department  Conducted annually  Implan software  County specific

35. ECONOMIC IMPACT USDA Economic Research Service Economic Brief Number 10, September 2007

36. UF FOUNDATION, INC.

37. UNIVERSITY REGISTRAR

38. WORKLOAD (ONLINE FACULTY ACCOUNTABILITY SYSTEM)

39. FACULTY REPORTING IN WORKLOAD

40. 1.Adobe InDesign – create the template 2.Microsoft Access – organize and combine the data 3.Microsoft Excel – format the data 4.Adobe InDesign – merge the data into template, creates single document with one page per county 5.Adobe Acrobat Pro – extract pages from file to create single county sheets 6.Adobe Acrobat Pro – edit single sheets to remove blank fields (i.e., address fields, commodities) 7.Post single large file (i.e., all counties) and individual county sheets to web CREATING COUNTY SHEETS

43. Advantages  Stakeholders have accurate quantitative data when advocating for IFAS Extension  State & County extension faculty have readily available data they can share with County administrators  Easy to use, simple to understand information available to all  University ties it to their own economic impact page Disadvantages  Limited amount of programs can be highlighted within the document  Development can be time consuming for administrators and staff  University data out of date, not maintained ROI ADVANTAGES & DISADVANTAGES

44.  How do we ensure outcomes are achieved over time?  In leadership teams?  By Extension professionals broadly?  Who should be tracking this? How?  How can cross-team collaboration be encouraged and/or rewarded?  What will make teams want to work together?  How can an evaluator influence leadership? Or should they? WHAT ROLE CAN EVALUATORS PLAY?