1. First-Year STEM Student Cohorts: Assessment and Best Practices Gary Hunt – Boise State University; Edmund Tsang – Western Michigan University

2. Session Structure Introduction – Tsang/Hunt Part I: Building Large Cohorts – Tsang Part II: Assessing Cohorts – Hunt Wrap-up – Hunt

3. Cohorts – Introduction Why do we create cohorts? – Improved retention Tinto: social and academic engagement – Social: organized social activities; informal – Academic: performance; interactions with faculty & staff – Campus: resource connections – Enrollment logistics Learning communities can help campuses with managing course offerings

4. Cohorts, examples Living Learning Communities Bridge programs Learning communities UROPs (undergraduate research opportunities programs)

5. Social Learning Theory: COPs Communities of practice are formed by people who engage in a process of collective learning in a shared domain of human endeavor: a tribe learning to survive, a band of artists seeking new forms of expression, a group of engineers working on similar problems, a clique of pupils defining their identity in the school, a network of surgeons exploring novel techniques, a gathering of first-time managers helping each other cope. In a nutshell: – Communities of practice are groups of people who share a concern or a passion for something they do and learn how to do it better as they interact regularly. http://www.ewenger.com/theory/

6. Communities of Practice COP are groups of people (community) Who share a concern or a passion for something they do (domain) And learn how to do it better (practice). We are trying to create COPs with most freshmen cohorts

7. Example: Engineering Residential Dormitory

8. Questions this session addresses – Logistics associated with large university creation of cohorts; – Assessment: how does an institution or program know if their cohort program is cost effective?

9. Building and Assessing Learning Community/Cohorts Since evidence shows learning community/cohort is a powerful strategy to support student success and retention Ideally should include as many students as possible  Logistics of building LC involving several hundred students (particularly if they have diverse academic preparation background)  WMU/CEAS Objectives – students in a cohort take the same 3-5 courses together in fall semester and 2-4 courses in spring semester of their first year Constrained to small number of students  Selection criteria are important – how to get the biggest impact  Avoid “false positive” results Highly Motivated  Moderately Motivated  Not Motivated (Well/Moderately/ (Well/Moderately/ (Well/Moderately/ Poorly Prepared) Poorly Prepared) Poorly Prepared)

10. Build Cohorts Involving ~400 Students at WMU Constraints at WMU/CEAS Entering 1 st -year CEAS students place into Algebra II (~25%), Precalculus (~30%), and Calculus I (~45%) 13 engineering and applied sciences program organized into 6 departments No common first-year curriculum in CEAS CCE, ECE, CHEG, PAPR have their own 1 st -semester course Mathematics, chemistry, physics courses outside CEAS and they do not want to “zero-out” any courses/labs What are the constraints on your campus?

11. Build Cohorts Involving ~400 Students at WMU Factors to consider in building cohorts Chemistry lab sections have capacities of 24 students/lab Math sections have capacity of 40 students/section Physics lab sections have capacities of 20 students/lab Engineering Graphics lab sections have capacity of 24 students/lab Technical Communication sections have capacity of 24 students/section What are the factors on your campus to consider?

12. Build Cohorts Involving ~400 Students at WMU Strategies to build cohorts at WMU # students/cohort = 24 based on section size of technical communication/engineering graphics lab, chemistry lab (take 12 seats from 2 labs) and other labs, and mathematics Importance of an anchor class Use admit and historic yield data, as well as MATH ACT sub- score to project # seats required Submit seat requests in January for summer orientation/fall registration; early October for spring registration; complete iterations in February/late October Keep departments informed of seat usage; release unused seats immediately What strategies would work best on your campus?

13. Build Cohorts Involving ~400 Students at WMU Examples of cohort course templates

14. Build Cohorts Involving ~400 Students at WMU Examples of cohort course templates

15. Build Cohorts Involving ~400 Students at WMU Examples of cohort course templates

16. Build Cohorts Involving ~400 Students at WMU Lessons learned Placing students in cohorts has become standard practice in CEAS summer orientation/registration since 2005, shifting focus of summer orientation from course registration to learning more about their programs and how to be successful Captures 85-90% of students (not placed in a cohort are students with many AP credits; athletes; students who opt out) Math, chemistry, and physics are very supportive of the practice  reduces uncertainty because they know which sections will make

17. Part II – Assessing Cohorts Gary Hunt, Boise State University Case Study 1: Stem Summer Adventure Any questions? Here is a summary of your responses… Which would you implement? What are the two most significant barriers to assessment? Discussion

18. Part II – Assessing Cohorts Gary Hunt, Boise State University Case Study 2: Research Any questions? Here is a summary of your responses… Which would you implement? What are the two most significant barriers to assessment? Discussion

19. Wrap-up: A specific cohort, and how it is being assessed.

20. What we did -- Stem Summer Adventure August 2012 5 day extended field program

21. Why? Goal: to increase first year retention, leading to increased STEM graduation rates The lever we are pulling: Build community, increase student engagement into campus Hypothesis: Incoming freshmen STEM students who participate in the summer adventure and the associated seminar class will be more integrated into the university than the average student.

22. Treatment: 5 days/4 nights of rafting, camping, social and curricular activities – Team building and social activities led by the outdoor program – Science curricular activities -- faculty lead with volunteers Weekly seminar class in fall – 15 one hour meetings include – Student success information Tours of labs and departments Social activities

23. How to measure success? NSSE-like question survey (national survey of student engagement) What is NSSE? – Nsse.lub.edu

24. How to measure success? Qualitative: student feedback in form of student videos and question responses Quantitative: NSSE question survey (national survey of student engagement) – BSU had administered NSSE – We had aggregate scores for the campus to compare with – Students in the summer adventure program took survey at the end of their first semester

25. Qualitative Feedback Typical student responses to: – What did you like the best about this trip? – What needs improvement? – Would you recommend this trip to a friend? http://www.youtube.com/watch?v=PAq6Cpzxqag&f eature=plcp

26. Quantitative results Selected NSSE questions were used to measure student engagement Statistical results showed that Cohort participants showed greater engagement. The result was significant. χ 2 (3, N = 12) = 8.9, p <.01

27. Discussion: What would you do to assess this program? What do you like about this program? What are you concerned about? What is your best take away from this session?