1. Advancing Competency-based Assessment in Entrepreneurship Michael H. Morris, Ph.D. Professor and N. Malone Mitchell Chair Head, School of Entrepreneurship Oklahoma State University

2. Assessment of What? (for us---student level, course level and program level) Integrated ‘E’ Programs Undergraduate Curriculum Scholarly Activity/Research Community-based Outreach Cross-Campus Initiatives Experiential Learning Campus-based Outreach Graduate Curriculum

3. Competency : definedCompetency : defined  Competence is a fuzzy concept useful in bridging the gap between education and job requirements (Boon and van der Klink (2002)  To have competencies is to possess the necessary attributes to perform competently (Burgoyne, 1988)  A characteristic of an individual that has been shown to drive superior job performance (Hartle, 1995)  Observable behaviors that superior performers exhibit more consistently than average performers (Klein, 1996) IMAGINE > BELIEVE > CREATE OKLAHOMA STATE UNIVERSITY

4. Competency : defined Competency : defined  Competencies include knowledge, skills, attitudes, values, behaviors and characteristics that people need to do a job successfully (Bryant & Poustie, 2001)  They correlate with job performance, can be measured against standards, and can be improved with training (Bryant & Poustie, 2001)  One can contrast areas of competence (aspects of the job which an individual can perform) with competency (a person’s behavior underpinning competent performance)  Competencies are connected to activities & tasks, subject to learning and developmental processes, and tend to be interrelated (Bergevoet, Mulder and Van Woerkum, 2005) IMAGINE > BELIEVE > CREATE OKLAHOMA STATE UNIVERSITY

5. Why teaching competencies matters…Why teaching competencies matters…  ‘competency’ is a term that allows for flexibility in adapting to diverse and changing organizational demands (Garman and Johnson, 2006)  From an hrm perspective, a competency is something that can be developed (Klarus, et al., 1999)

6. Management vs. EntrepreneurshipManagement vs. Entrepreneurship  Management: Getting people together to accomplish desired goals and objectives. It comprises PODSCORB  Entrepreneurship: Recognizing and exploiting opportunity through new combinations. But what does it comprise? IMAGINE > BELIEVE > CREATE OKLAHOMA STATE UNIVERSITY

7. The entrepreneurial experience…The entrepreneurial experience… Limited Sense of Control Loneliness AmbiguityDejection Stress Freedom Exhilaration Uncertainty ResponsibilitySelf-reliance Learning Adaptation Discipline Change IMAGINE> BELIEVE > CREATE OKLAHOMA STATE UNIVERSITY

8. Does the context matter? Does the context matter?  Start-up  Small or family business  Buying a business  Franchising  Corporate entrepreneurship  Social entrepreneurship  Public sector entrepreneurship  Entrepreneurship in a discipline (art, engineering) IMAGINE > BELIEVE > CREATE OKLAHOMA STATE UNIVERSITY

9. Or is the behavior universal?Or is the behavior universal?  Innovative, risk-taking, proactive behaviors  Acting upon opportunity  Unique combinations  Creating something from nothing (or from something very different)  A mindset that is both attitudinal and behavioral …regardless of the context IMAGINE> BELIEVE > CREATE OKLAHOMA STATE UNIVERSITY

10. And so…And so…  Perhaps the managerial requirements or competencies needed to succeed in social entrepreneurship or corporate entrepreneurship or new venture creation differ  But do the entrepreneurial competencies differ? Perhaps not! IMAGINE> BELIEVE > CREATE OKLAHOMA STATE UNIVERSITY

11. Stage 1 Methodology: Delineating Competencies  Two expert panels  Three waves  Survey Monkey  Produced total of 167 competencies  Split into two major groups: managerial and entrepreneurial  Eventually arrived at 13 core entrepreneurial competencies

12. The key competencies (note their inter- dependencies)  Recognizing Opportunity  Assessing Opportunity  Vision/Seeing the Future  Creative Problem-solving  Resource Leveraging/Bootstrapping  Mitigating and Managing Risk  Planning/Modeling When Nothing Exists  Innovation---Value-driven New Product and Concept Development  Building and Managing Networks  The Ability to Maintain Focus Yet Adapt  Action Orientation/Implementation  Tenacity/Perseverance  Ability to Learn from Experiences IMAGINE > BELIEVE > CREATE OKLAHOMA STATE UNIVERSITY

13. Distinguishing Sample Competencies managerial  Organizing  Team building & Staffing  Communicating  Budgeting  Controlling  Motivating  Planning  Directing  Operating  Assessing entrepreneurial  Recognizing Opportunity  Assessing Opportunity  Creative Problem-solving  Resource Leveraging  Guerrilla Skills  Mitigating and Managing Risk  Planning When Nothing Exists  Innovation---Products, Services, Processes  Building & Managing Social Networks  Adaptation while Focusing  Implementation of Something Novel or New

14. So we are doing bothSo we are doing both  Developing managerial competencies in the business school  Developing entrepreneurial competencies in the entrepreneurship program  Both are needed for success in an entrepreneurial context, although the relative importance of a given competency will vary IMAGINE> BELIEVE > CREATE OKLAHOMA STATE UNIVERSITY

15. How do we teach these competencies?How do we teach these competencies?  Lectures  define each competency  illustrate each  strategies for managing each  examples  relate to other competencies and learning points  repetition  Experiential learning in the classroom  Experiential learning outside the classroom

16. Teaching: experiential learningTeaching: experiential learning  Cases  Student incubators  Tech commercialization teams  Small business consulting projects  Entrepreneurial audits  Marketing inventions  Creativity field experiences (e.g., the Lowe’s experience)  Simulations  Entrepreneurs in the classroom  Interviews of E’s  Unique internships  Mentorships and job shadowing  Role plays (VC’s, family firms, etc.)  Business models  Business plans and competitions  Social entrepreneurship projects in the community

17. Linking experiential learning to outreachLinking experiential learning to outreach  Native American Entrepreneurship Academy  Community Microcredit Fund  Entrepreneurship Empowerment in South Africa  Inner City Engagement  Disabled Veterans Bootcamp  OSU Entrepreneur’s Bootcamp  Riata Business Plan Competition  Entrepreneurial Mentors Program  OSU Technology Commercialization Initiative  Dilemmas and Debates  Enterprise Creation Competition, a national business plan competition  Women Igniting the Spirit of Entrepreneurship  The Experiential Classroom  Riata Entrepreneurial Internships  Distinguished Lectures & Workshops  Commercial Test Kitchen

18. Connectivity is keyConnectivity is key ClassroomOutreach Experiential Learning

19. Integrate across the E curriculum ContextsCompetencies Start-up Ventures Early growth firms Family Businesses Rapid Growth Ventures Corporate Entrepreneurship Entrepreneurship within Professions & Disciplines Non-Profit & Social Entrepreneurship Public Sector Cultural Entrepreneurship Academic Entrepreneurship Opportunity Identification Opportunity Assessment Resource Leveraging Guerrilla Techniques New Product- Service- Process Development Risk Management Creativity Adaptation Social Networking Implementation Skills Novel planning

20. Measuring competencies—some caveats Luken 2004; Le Diest and Winterton, 2005  the definition of a given competence is not a homogeneous definition  competencies are not stable  competence assessments are always subjective  competence assessments are based on individuals, whereas the definition of the competence concept tries to include the context as well  Competence or skill is tied to an individual, but the individual’s competence may vary depending on context  the competence concept includes capacity, whereas it is also important to look at actual performance

21. Sub-dimensions in mastering a competency  Knowledge and Understanding: what do you need to know about resource leveraging  Attitude/Affect and Self-Awareness: what do you need to think, believe and feel about resource leveraging?  Skills and Behaviors: what do you need to be able to do in terms of resource leveraging? These are all learning outcomes We can do more not just in terms of conveying knowledge, but in all three areas, especially to the extent that we stress experiential learning IMAGINE> BELIEVE > CREATE OKLAHOMA STATE UNIVERSITY

22. Each competency requires a definition and level of proficiency  Definition: What do we mean by the competency?  Level of Proficiency: What must the student be able to demonstrate in terms of knowledge, skills, capabilities and attitudes he competency to indicate mastery of the competency?

23. An illustrationAn illustration Example: Risk management Definition:The ability to identify relevant risks surrounding an entrepreneurial action and systematically mitigate those risks. Level of Proficiency: 1. Understands key types of risks 2. Can identify principle risks surrounding a given entrepreneurial action 3. Can prioritize risks based on magnitude and probability of loss 4. Is able to develop specific actions to -stage the risk -share the risk -reduce the risk IMAGINE> BELIEVE > CREATE OKLAHOMA STATE UNIVERSITY

24. Sample ways to break it downSample ways to break it down  Knowledge  Nature of risk versus uncertainty  Dimensions of risk  Categories of risk  General techniques for mitigating risk  Attitudes/values  Willingness to assume moderate levels of risk  Belief that risk is manageable  Sense of association between risk level and potential return  Behaviors/Skills  Ability to estimate risk  Ability to isolate risk  Ability to moderate level of risk

25. A second illustrationA second illustration Example: Opportunity identification Definition:The ability to specify unrecognized or unfilled gaps in the external environment creating an opening for a new product, service or process. Level of Proficiency: 1. Understands general sources of opportunity 2. Is capable of scanning the environment to identify emerging patterns & trends, competitor shortcomings, unutilized resources & unmet needs 3. Can connect an opening in the environment to a specific target audience with a need IMAGINE> BELIEVE > CREATE OKLAHOMA STATE UNIVERSITY

26. Sample ways to it down…Sample ways to it down…  Knowledge  Key sources of opportunity  Major types of opportunity  The nature of opportunities  Four ways in which opportunities are identified  Understanding of specific opportunity generation techniques  Attitudes/values  Curiosity about why things work a certain way  Value one places on being alert to opportunity  Openness to being exposed to diverse and changing situations  Behaviors/Skills  Ability to draw associations  Ability to grasp and hold onto ideas as they occur to us  Ability to assess customer needs  Response to a failure (e.g., elevator pitch loss)  # of opportunities generated  Novelty of ideas generated

27. Measurement approachesMeasurement approaches  Pre- and post- measures using rating scales  Judging experiential project portfolio  Behavioral event interviews  In class assessments tied to exercises  Student diaries or registers  Peer assessments  Self-assessments at end of program  Behavioral assessments after graduation (see also Bird, 1995) IMAGINE> BELIEVE > CREATE OKLAHOMA STATE UNIVERSITY

28. Examples of existing scales that tie inExamples of existing scales that tie in  Innovativeness: Kirton Adaption Innovation Inventory  Networking: Hatala Network Accessibility Scale  Ambiguity Tolerance: Budner’s Tolerance for Ambiguity Scale  Adaptability: Haynie and Shepherd’s Cognitive Adaptability Scale

29. Setting the standard for a rubricSetting the standard for a rubric  Criterion-referenced evaluation: student performance is assessed relative to standards set by the discipline or entrepreneurship faculty  Norm-referenced evaluation: students are evaluated on the basis of comparisons to other students

30. Establishing normsEstablishing norms  We have no norms  Benchmarks must be established  Suggest we initially evaluate students relative to one another  Over time we might create benchmarks using successful entrepreneurs  But----are certain competencies more critical for success in certain types of contexts?  Is our focus less on achieving some absolute level on a competency ---- or more on showing improvement relative to where a student started?  Competency is a process of continual development through one’s life---not riding a bike---fades without practice

31. Sample template for scales (Mertler 2001) Beginning 1 Developing 2 Accomplished 3 Exemplary 4Score Criteria #1 description Criteria #1 description Criteria #1 description Criteria #1 description Total Score = _______ Score at Program Outset _______ Average Student Score ________

32. Some other tipsSome other tips  Match measurement devices to instructional goals  Do not rely on a single measure of a competency  Teacher judgment is the primary means for assessing competency  Certify competency in progressive stages  Get the students on your side  Allow for easy adjustment of competency measures as content changes  Allow a feedback loop where measurement of competencies is used to modify course content and curriculum design -SeeTyo (1980)

33. “ Be the change you wish to see in the world.” -M. Gandhi