Developing a Comprehensive Learning Outcomes Assessment Strategy for Adult Learners Cindy MacMillan& Susan Davenport

2 Agenda Introductions Characteristics of Thomas Edison State College Challenges in an adult learning environment

3 Introductions Susan Davenport, Dean of Heavin School of Arts and Sciences Cynthia MacMillan, Director of Outcomes Assessment and Accreditation Audience –Who’s here? –Where you are from? –Why this session?

4 Why Learning Assessment? Is it really all about accountability? Accreditors Students Mission You

5 What are the challenges assessing learning in an adult environment? Heterogeneous population with different needs and skills Significant learning that the student brings to the learning environment Access to students for assessment

6 Where to Start Phase One – Define Learning Outcomes at Institutional and Programmatic Levels Phase Two-- Operationalize the learning outcomes Phase Three – Implementation of assessment (finalize assessment matrix) Phase Four – Data collection and Analysis Phase Five -- Recommendations and transformations made in response to Phase Four data

7 Where to start! (cont.)

8 Phase One Define Learning Outcomes at Institutional and Programmatic Levels Buy In Strength in diversity Define and Describe Goals and Process –Institutional, Programmatic, Course

9 Thomas Edison State College Institutional Outcomes 1. Information Literacy The Information literate graduate will demonstrate the ability required to recognize when information is needed and has the ability to identify, locate, evaluate and use effectively and appropriately the information needed. 2. Technological Competency Graduates will be able to solve problems and perform tasks using appropriate technological tools. 3. Responsible Ethical Leadership Graduates will be able to articulate in their own words the ethical principles they have discovered and encountered in their academic discipline or profession. Graduates will recognize and articulate actual ethical dilemmas witnessed or experienced. 4. Scientific Reasoning Thomas Edison State College graduates will be able to analyze issues facing citizens by distinguishing scientific approaches from other ways to frame problems and issues. 5. Diversity/Global Literacy The Thomas Edison State College graduate will demonstrate an understanding of globally diverse perspectives, cultures, values, events, trends and issues.

10 Thomas Edison State College Institutional Outcomes (contd.) 6. Quantitative Reasoning/Literacy Thomas Edison State College graduates will be quantitatively literate. They will be able to understand and apply mathematical concepts and skills to inform personal and professional decision-making. 7.Written and Oral Communication Thomas Edison State College Graduates will be able to write and speak with proficiency. 8. Critical Analysis & Reasoning Thomas Edison State College graduates will be able to demonstrate critical thinking and problem solving skills and are able to analyze content, discover meaning or significance, draw conclusions, and make assessments. 9. Life Long Learning Graduates will demonstrate a commitment to lifelong learning.

11 Phase Two Operationalize the Learning Outcomes Indirect versus direct measures Quantitative versus qualitative measures Triangulation Low versus High Stakes Assessment Baselines and Benchmarks

12 Phase Two Continued Methods we are using – Standardized Assessment – ETS Proficiency Profile, SAILS Graduate Survey, Adult Learner Inventory Capstones Embedded Analysis Historical Data Review Survey ETS Major Field Test E-Portfolios

13 Example: Assessment Matrix for Program Level Learning Outcomes Bachelor of Science in Human Services outcomes: Program Element which addresses this outcome Measures (indicate direct or indirect)Benchmark/ Target (acceptable and ideal if appropriate) Interpret and critically analyze the research in the professional track Course in Professional track final project/paperA 70% earn a grade of B+ I 80% earn a grade of B+ SOS 492 Research Methods final research paper A 70% earn grade of B+ I 80% earn grade of B+ ETS Proficiency Profile Std. Test Critical ThinkingA 60% achieve proficient or marginally proficient I 80% achieve proficient or marginally proficient Apply theory to professional practiceCapstonefinal paperA 70% earn grade of B+ I 80% earn grade of B+ Apply knowledge of the specific skills, techniques, and agencies necessary to serve client populations PSY 331 Intro Counseling final examA 70% earn grade of B+ I 80% earn grade of B+ Capstonefinal paperA 70% earn grade of B+ I 80% earn grade of B+ Graduate SurveyItem 7-aA 80% respond positively I 90% respond positively Apply knowledge of cultural diversity as it relates to the field of Human Services. SOC 322 Cultural Diversity in the US final examA 70% earn grade of B+ I 80% earn grade of B+ Capstonefinal paperA 70% earn grade of B+ I 80% earn grade of B+ Apply theories of management as it relates to Human Services MAN 301 Principles of Management final examA 70% earn grade of B+ I 80% earn grade of B+ Capstonefinal paperA 70% earn grade of B+ I 80% earn grade of B+

14 Phase Three Implementation of Assessment Be realistic in scope Not everything needs to be assessed each year Be pragmatic in tying plan to funds Benchmarking –Internal –External Rubric elements

Information Literacy Data Collection Sample 15

Example: Data Collection for Human Services Degree 16 Bachelor of Science in Human Services outcomes: Benchmark/ Target 2010 results Interpret and critically analyze the research in the professional track 70% earn a score of 3 or above on a 4 point scale on rubric by reviewed independent consultant Above target Apply theory to professional practice70% earn a score of 3 or above on a 4 point scale on rubric by reviewed independent consultant Below target Apply knowledge of the specific skills, techniques, and agencies necessary to serve client populations 70% earn a score of 3 or above on a 4 point scale on rubric by reviewed independent consultant Below target Apply knowledge of cultural diversity as it relates to the field of Human Services. 70% earn a score of 3 or above on a 4 point scale on rubric by reviewed independent consultant Below target Apply theories of management as it relates to Human Services 70% earn a score of 3 or above on a 4 point scale on rubric by reviewed independent consultant Above target

Phase Four and Five– Analysis and recommendations 17

Information Literacy - SAILS Recommendations Continue to evaluate student learning outcomes in the area of Information Literacy using the SAILS. Embedded SAILS in an upper level course for general populations. Identify and implement additional methods to assess the outcome area not assessed by SAILS. Include the SAILS in a few more courses to ensure that all degree programs of the College are included in this informational literacy assessment. Target the SAILS administrations to students per year in order to facilitate longitudinal analysis in the future. 18

Information Literacy – ENC-102 Recommendations Expand the rubric elements to include one on grammar and spelling. Expand the data collection efforts in ENC-102 to include one section per semester. Expand the roll of the lead mentor to include collecting reliability data. Expand the use of Waypoint in other schools, courses and assignments. Explore viable options to communicate details of Waypoint use with mentors. Explore ways to minimize the use of instructional design resources. Seek efficiencies such as sharing rubric elements across schools. 19

20 Human Services – Analysis and recommendations Analysis of Human Services results led to the following curricular changes: 1. Modify the program description to more accurately describe the program and its objectives. 2.Modify the program outcomes to align with objectives in program by including diversity and theories of management. 3.Modify the general education requirements to 60 credits to align with the other degree offerings within the College. Require a three credit Ethics course under Humanities.

21 Phase Four and Five– Analysis and recommendations (contd.) 4. Modify the Core Requirements. Based on guidelines from national accrediting bodies (i.e. The National Human Services Organization and the Council for Standards in Human Services Education), the core requirements were replaced with the following areas: Theoretical Foundation, Intervention, Client Populations, Research and Mgmt. This improves the quality of student learning outcomes and ensures that the program objectives are appropriate and measurable. 5.Modify the Practicum to a capstone course and objectives to make the practicum an effective instrument for assessing the degree outcomes. 6. Change the grading of the capstone from Credit/No Credit to a letter grade. This will ensure a more accurate assessment of the outcomes of the course. A benchmark of “B+” was recommended.

22 Your best practices, lesson learned and questions Please share your feedback...

Thank you for your time and interest!