1. Assessment and Evaluation of CAREER Educational Components monica.devanas@rutgers.edu Center for Teaching Advancement and Assessment Research

2. Project Description a description of the proposed research project, including preliminary supporting data where appropriate, specific objectives, methods and procedures to be used, and expected significance of the results; including plans to evaluate their impact on students and other participantsa description of the proposed educational activities, including plans to evaluate their impact on students and other participants; a description of how the research and educational activities are integrated with one another; and results of prior NSF support, if applicable.

3. Education Activities designing innovative courses or curricula supporting teacher preparation and enhancement conducting outreach and mentoring activities to enhance scientific literacy or involve students from groups that have been traditionally underrepresented in science researching students' learning and conceptual development in the discipline incorporating research activities into undergraduate courses providing mentored international research experiences for U.S. students linking education activities to industrial, international, or cross-disciplinary work implementing innovative methods for evaluation and assessment.

4. Educational Program Review Education Program Goals –3-5 overarching goals Integrate research areas into curriculum Increase applicant pool to your program –via Summer internships for high school students Connect with international scholars

5. Cycle of Assessment/Evaluation NSF Handbook for Project Evaluation, page 4

6. Cycle of Assessment/Evaluation NSF Handbook for Project Evaluation, page 4 Pre/post test Survey students Faculty Feedback Assessment Committee Pre/post test Survey students Faculty Feedback Assessment Committee Update field New course State–of--Art Content in class and labs Modifications to New Course

7. Measureable Outcomes NSF Handbook for Project Evaluation, page 18

8. Measureable Outcomes New Course NSF Handbook for Project Evaluation, page 18

9. Kinds of Assessments Direct Assessments Quantitative Assessments Indirect Assessments Qualitative Assessments

10. Kinds of Assessments Direct Assessments Pre and posttests Course-embedded assessment (homework assignment; essays, locally developed tests) Comprehensive exams National Major Field Achievement Tests GRE subject exams Certification exams, licensure exams Senior thesis or major project Portfolio evaluation Case studies Reflective journals Capstone projects Internship and clinical evaluation External examiners/peer review Grading with criteria or rubrics Indirect Assessments Departmental survey Exit interviews Alumni survey Employer survey Focus groups Job placement statistics Graduation and retention rates Percentage of students who study abroad

11. Kinds of Assessments Quantitative Assessments Surveys/Questionnaires Pre/post Tests Existing Databases Statistical Analysis Qualitative Assessments Observations Interviews Focus Groups Non-statistical

12. Kinds of Assessments Direct Assessments Direct Assessments Pre/post test, student work Quantitative Assessments Statistical analysis of surveys Student demographics Indirect Assessments Exit Interview/survey, focus group Qualitative Assessments Interviews, observations Anecdotal examples

13. Educational Program Activities Education Program Goal #1 Activities –Integrate research areas into curriculum –New course »Course portfolio, syllabus, student projects »Pre/post tests of knowledge, skills and attitudes »Surveys of students »Focus groups »Feedback from colleagues in department »Enrollment increase over time »Student success in subsequent courses, career options

14. Educational Program Review Education Program Goal #1 Activities –Integrate research areas into curriculum –New course »Course portfolio, syllabus, student projects »Pre/post tests of knowledge, skills and attitudes »Surveys of students »Focus groups »Feedback from colleagues in department »Enrollment increase over time »Student success in subsequent courses, career options

15. Educational Program Activities Education Program Goal #2 Activities –Increase applicant pool to your program –Enrollments from past years compared to post award –Demographics of applicants »Underrepresented »SAT’s, GPA’s, GRE’s –Student projects –Surveys of students, exit surveys, –Feedback from faculty

16. Educational Program Review Education Program Goal #2 Activities –Increase applicant pool to your program –Enrollments from past years compared to post award –Demographics of applicants »Underrepresented »SAT’s, GPA’s, GRE’s –Student projects –Surveys of students –Surveys of students, exit surveys, –Feedback from faculty

17. Educational Program Activities Education Program Goal #3 Activities –Connect with international scholars –Collaborative projects –Joint publications, symposia –Inter-institutional visits, graduate student exchange »Interviews, surveys, student work »Retention, success in advanced work »International Post-graduate work –Measureable outcomes at both sites

18. Educational Program Review Education Program Goal #3 Activities –Connect with international scholars –Collaborative projects –Joint publications, symposia –Inter-institutional visits, graduate student exchange »Interviews, surveysstudent work »Interviews, surveys, student work »Retention, success in advanced work »International Post-graduate work –Measureable outcomes at both sites

19. Types of Evaluation NSF Handbook for Project Evaluation, page 8

20. Types of Evaluation NSF Handbook for Project Evaluation, page 8 Formative Implementation of Modifications Progress Reports Implementation of Modifications Progress Reports Summative FINAL REPORT

21. Assessment Oversight Internal Assessment Committee –PI –Departmental colleagues –Other colleagues not affiliated with project –Meet regularly to review process and formative assessments –1-2/semester External Reviewer – Assessment “expert” – Not affiliated with institution, or project – Consult on Assessment Plan for grant – Examine implementation process during grant – Final Summative Report – 1/year

22. Types of Evaluation NSF Handbook for Project Evaluation, page 8 Formative Assessment Committee External Reviewer

23. Best Practices for Assessment and Evaluation Have an assessment plan Balance Direct/Indirect Quantitative/Qualitative Assessment committee External Reviewer ? Collect everything you can Talk to every one you can Present/publish research & education activities Evaluate Programs in Final Report Apply for more grants