1. CIRTL Network Meeting www.cirtl.net/networkgroup June 19, 2014 11:00-12:30 ET/ 10:00-11:30 CT/ 9:00-10:30 MT / 8:00-9:30 PT This meeting will be recorded Begin by running the Audio Setup Wizard Select : Tools>Audio> Audio Setup Wizard or select the audio wizard icon Backup Call-in: Call-in Number: 1 (571) 392-7703 Passcode: 931 749 952 188

2. Agenda Announcements Integrating Cognitive Science with Innovative Teaching in STEM Disciplines Update Cross-Network Programming Update - possible funding opportunity Video: Joe Hardcastle’s Teaching as Research Project – Boston University (5 min) Topic 1: Local Program Evaluations Cornell University University of Colorado at Boulder

4. Cross-Network Update

5. www.cirtl.net Fall Cross Network Courses Teaching as Research I (first of two semester sequence; students can take one or both) The College Classroom (MOOC) Diversity in the College Classroom (tentative, need co- instructor) Training In Teaching and Learning Modules (or mini course) (titles tentative)

6. www.cirtl.net Fall Cross Network Mini Courses and Other Programming Mini Course: The Active Classroom: Supporting Successful Small Group Learning Mini Course: Student Centered Course Design Mini course: Developing a Scholarly Approach to Teaching (tentative) Journal Club, CIRTL Reads Online Learning Community for International STEM students (titles tentative)

7. www.cirtl.net 2014-15 CIRTLCast Series Themes for the one-month blocks will include: 1.Diversity Issues in STEM Education 2.Financial Literacy 3.Educational Innovations for the STEM Classrooms (2 months) 4.Job search 5. Surviving the First Year as Faculty Member 6.One other, TBA As soon as we get all the dates, titles and descriptions for the series settled, we will be soliciting speakers and panelists for the sessions. Watch for this solicitation in upcoming weeks. As soon as we get all the dates, titles and descriptions for the series settled, we will be soliciting speakers and panelists for the sessions. Watch for this solicitation in upcoming weeks.

8. www.cirtl.net Upcoming Cross Network Dates Mon, July 7, 2014: Fall 2014 Course Offerings Posted Online Mon, July 21, 2014: Fall 2014 Course Registration Opens Mon, August 18, 2014: Fall 2014 Annual Event Calendar Posted Mon, September 22, 2014: Spring 2015 Cat 1 Proposals Due Dates posted on the Cross Network Page at: http://www.cirtl.net/networkgroup/Lclinks

9. Funding Opportunity Update

10. Joe Hardcastle's Teaching as Research Project - Boston University Vimeo Link:

11. Cornell University CIRTL Programs on Mentor Training and Their Evaluation 1.Building Mentoring Skills Lunchtime Workshop Series (discussed in May 2013) 2.REU Mentor Trainings (initial training and optional mid- summer check-in lunch) 3.Developing video case studies 4.High-engagement experiential programs (today’s focus)

12. A Coupled Graduate-Undergraduate Course in Field Research Mentorship “Graduate training programs for future faculty must build a comfort level for engaging with students in the field.” Middendorf & Pohland (2014)

13. Overall Learning Goals Undergraduate mentees will: Develop an understanding of and appreciation for natural history Build skills and confidence to propose, develop and test hypotheses and to communicate results Practice vital collaboration and cooperation skills needed for research and the job market

14. Learning Goals Graduate mentors will: Empower mentees to design and confidently execute independent, authentic, inquiry-based research projects Prepare for logistical challenges and to understand and capitalize on student diversity Manage personal dynamics with co-mentors and between mentees

15. Undergraduate Experience: Bi-weekly course meetings during semester to research and present on aspects of the study system 10-day trip to biological field station immediately after spring semester Lead TA (designing research/evaluation project) Training Structure

16. Graduate Mentor Experience: Monthly meetings with professors to address learning objectives. Attendance at CU-CIRTL Building Mentoring Skills workshops Field Trip: Facilitated “exploration and inquiry” period Each mentor assigned four students Mentored for design and execution of group research project Ongoing supervision from professors and nightly group discussion (learning community) Training Structure

17. Questions?

18. Description of Evaluation Plan Implementation: Pre- and post-course surveys (separate undergraduate and graduate instruments) –Qualitative and quantitative questions –Administered online TA observation and informal interviews in the field Potential for analysis of student artifacts (research posters) Goals: Mentors: Assess mentorship skills development, attitudes toward field mentorship Undergraduate: assess research skills development, provide feedback for mentors Both: Get feedback on pilot course for future development and facilitator guide

19. Mentor Attitudes: Course “increased the likelihood that they will seek out field-based mentoring opportunities in the future” (for 3/3 mentors) “I feel significantly more confident and excited about field mentoring. This experience has convinced me that this should be something I try to integrate into my future career in some capacity.” Mentor Skills: Skills where self-assessed proficiency went from “low” to “medium/high” (for 3/3 mentors) Building mentee independence Building mentee confidence Developing strategies to deal with mentoring challenges Results

20. Mentor Confidence: Confidence increased by one or more categories for the following items: Empowering mentees to make their own observations and suggest questions and hypotheses (3/3 mentors) Refocusing mentee attention if necessary in order to complete a specific task (3/3 mentors) Dividing roles and responsibilities between co-mentors (3/3 mentors) Managing mentee interpersonal group dynamics (1/3 mentors) (options: none, low, moderate, high) Results

21. Open-Ended Questions “I had a conflict about choosing methods with mentees…. After slightly guiding their conversation towards the downsides of the method… I put their decision in the context of peer review. They then understood that if taken seriously and reviewed by scientists, their decisions would reflect bias and a flaw in the experiment.” Did you experience any kind of conflict between yourself and a mentee or have concerns about their actions, attitudes or progress? What steps did you take to address this?

22. Questions?

23. Evaluation Challenges and Suggestions Designated time period for surveys to encourage timely participation More targeted questions regarding career perceptions and changes for undergraduates Change trip structure so that mentors arrive early and have field time with professors Advice on further alignment to CIRTL pillars and the new learning outcomes is welcome.

24. Resources and References Middendorf, G. & Pohlad, B.R. (2014) Ecoliteracy for ecology and ecologists: eroded underpinnings. Frontiers in Ecology and the Environment, 12 : 194-195. Course manual (in prep): Empowering tomorrow’s field instructors: A coupled graduate-undergraduate course in field research mentorship. Cornell University. Feedback for inclusion in course manual is welcome. Please address to Fiona Soper (fms46@cornell.edu)

25. Questions?

26.  TIGER CIRTL Evaluation Report 2013-14 Laura L. B. Border, Director, Graduate Teacher Program C0-PI CU Boulder, CIRTL Project

27. Program & Purpose  TIGER 2  TIGER 2 is a workshop series designed to highlight effective participation on research teams and professional development for RA’s.

28. TIGER 2 Workshop Content  5 Workshops on Aspects of Teamwork  3 Workshops on CIRTL Pillars  3 Workshops on Research in General  1 Workshop on Assessing Student Learning  1 Workshop on Preparation for Job Market (Research Statement)

29. Questions?

30. Evaluation Questions 1. Who and from which departments participated? 2. How many times did an individual return? 3. Did the participants report that they learned what was presented? 4. Did the participants have an idea about how they might apply the information presented? 5. Were participants satisfied or dissatisfied with the workshop? 6. Which workshops were most effective and why?

31. Features Measured 1. Participation 2. Dosage 3. Learning 4. Application 5. Satisfaction

32. Tools Used  Feedback Forms  Sign-in Sheets with demographic information  FileMaker Pro

33. Data Analysis  Spreadsheets  FileMaker Pro  Tracking and analysis of student responses to questions on feedback forms

34. Findings: Quantitative  12 workshops over two semesters  61 participants, 35 individuals  12 attended more than one workshop  13 STEM Departments represented  84% of participants returned Feedback Forms  6 individual STEM Faculty presented  Average Satisfaction Score: 4.91 (6=Excellent)

35. Dosage

36. Satisfaction  Average Satisfaction Score (6=Excellent) Fall: Workshop overall: 4.54 Fall Presenter Rating: 5.15 Spring: Workshops overall: 5.27 Spring Presenter Rating: 5.33

37. Effectiveness of Marketing  Where did you hear about the workshop?  GTP Website28  CIRTL Website 0  CU Calendar 4  Lead Graduate Teacher17  Department listserv 5  GTP Poster in Department 3

38. Feedback: Qualitative  What was most helpful part of this workshop?  What was least helpful part of the workshop?  Describe how you might apply the information in your classroom?  Would you recommend this workshop to a friend?  Suggested topics? Suggested Speakers?  Where did you hear about the workshop? (GTP Website, CIRTL Website, CU Calendar, Lead, Department, GTP Poster in Department)

39. Most Helpful

40. Least Helpful

41. Learning  Measured by statement, “I learned”:  How to integrate research and teaching  How to assess student learning  How to assess teaching  What I need to work on  What I need to articulate  What to keep in mind for proposal defense

42. Application

43. Impact & Effectiveness  Most effective workshops were:  Working on a Research Team  Writing Your Research Statement for a Teaching Portfolio  Most impact:  Increased participation in TIGER activities  Content of TIGER 2 was deemed highly relevant  Collaborative workshop environment was highly valued  Self-awareness & professional development was noted by

44. Potential Change in Evaluation  What changes, if any, might you make to improve or extend your CIRTL-related evaluation activities?  We would like to follow-up with participants via survey or interview.  Build in a focus on inter-disciplinarity.  Add a specific question on “What did you learn today?” to our Feedback Forms

45. Formative Improvement Possibilities  How might these findings be useful to CIRTL programs?  Emphasize need for shared discussion in workshops.  Emphasize need to highlight professional development aspects of programming.  Emphasize that analyzing your own data according to a rubric pulls out information that you might have not noticed.

46. Questions?