1. Randomizing Nursing Students into a Flipped Experience

2. Susan Harrington, Nancy Schoofs, Melodee Vanden Bosch, Cynthia Beel-Bates, Kirkhof College of Nursing Kirk Anderson, Statistics Department Concurrent Sessions A 10:45 - 11:45 Session A6 Room 410 EC

3. Background In 2006, KCON had its first cohort of accelerated second degree students (10-week semester) Teaching team decided to teach without lecture in the classroom – Case studies – Video-taped lectures – Practice questions – Discussion Students were successful, but asked for lecture after two years of this format

4. Background, continued In summer 2012, had our first class in new curriculum – Lecture only For fall 2012, decided to try flipped classroom – Video-taped lectures – Case studies, games – Practice questions, clickers – Discussion, simulations

5. Educating Nurses: A Call for Radical Transformation (Benner, 2010) Renovation in nursing education are mandatory and imminent Strive to create effective teaching strategies to assist in integrating knowledge-acquisition (classroom) and knowledge-use (clinical practice) Assimilate nursing science with clinical experience o Embed concepts in clinical practice o Engage students with active learning

6. Limited quantitative research Increased Interest in Active Learning

7. Speak Up National Research Project (Fall, 2012) Estimated that only 3% of the teachers in the U.S. know about or "do" flipped learning 27% of principals indicated their teachers wanted to try it this year!

8. Randomize, Randomize, Randomize Increase the level of investigative rigor by randomizing the 84 students o Winter 2013 o one of two teaching methods: (1) traditional / lecture driven or (2) flipped classroom. o collection of both qualitative and quantitative information to enhance and document the experience with hard data

9. HRRC Exempt status: normal educational practices in routine educational settings. Reason: KCON is engaged in continuous quality improvement concerning teaching effectiveness and routinely institutes innovative techniques in the classroom

10. Consent Requirement Waived Research presents minimal risk of harm to students and involves no procedures for which written consent is normally required outside of the research context. Consent document would be the only potential record linking the subject and the research, which may potentially result in a risk of harm from a breach of confidentiality.

11. FTLC Grant Pew Scholar Teacher Grant: 13-253 “A Comparison of Two Pedagogies: Flipped Versus Traditional Classroom in Nursing Education” Purpose of grant to fund increased workload of teaching team

12. Students Notified (Prior to identifying which group) Blackboard email to incoming class Investigate / compare two different teaching methods Asked to participate / will be randomly assigned Separate Blackboard sites / access to only one of the sites Assurance that professors have taught with this methodology in past Both teaching methods have been shown to be effective / student success is our priority No potential risk or harm to participants Assessments include: grades and surveys / confidentiality, de- identified surveys

13. Student Concerns (email “uproar”) “Guinea Pigs” do not want to be a part of experiment or research Need our consent / inconsiderate and unethical Prefer not to participate / respectfully decline We were not asked / or given a choice / inconsiderate Not my style of learning Seems last minute, ill-prepared / not fair Look at it from a student’s perspective / too much work Advocating for self and classmates Taking advantage of us / powerless Detrimental to learning / environmental context-dependent memory for better recall

14. Address Student Concerns Available to address concerns to student: set up time and place Many months of preparation Submitted and approved by GVSU’s HRRC. KCON routinely institutes innovative techniques in the classroom FERPA: no consent / Individual academic records are not being analyzed Nursing is an application science: regardless of technique, study time will be increased next semester and application will be encouraged/promoted/advocated Advocates for proactive learning E VERYTHING - except delivery of lectures - will be identical

15. Talking Points Consistency at presentation to students / first day of class / Written script Exams : same exam at the same time for both groups Quizzes : online; same quizzes - due the day before class Classroom time : Flipped classroom: activities such as case studies, multiple choice questions, care maps, care plans, skits, games Traditional classroom: straight lecture and some activities. Preparation for class: Flipped classroom: listen to the Camtasias which are the same lectures that the Traditional students will be getting in class; read notes (outlines or PowerPoint slides) and read the textbook readings. The Camtasias can be listened to multiple times Traditional classroom: read their outlines or PowerPoint slides and read the textbook readings

17. (Traditional format)

19. Out-of-class Flipped Class In-class Flipped Class Bloom’s Taxonomy

20. Methods: Study Design Experimental design Randomized convenience sample of 82 students – 41 students in flipped classroom – 41 students in traditional classroom

21. Methods: Sample & Setting 82 undergraduate nursing students Kirkhof College of Nursing (KCON) 2 nd semester of program (3 rd year of college) 14 weeks Winter semester of 2013

22. Procedures Two separate Blackboard sites Two separate classrooms – Same time and day of the week (Monday & Tuesday) – Same content on same date – Same amount of classroom time (1 – 2:50 pm)

23. Procedures (continued) 4 investigators – Each taught content in their area of expertise Just-in-time quizzes – Same online questions – 5 per classroom session 3 exams – Same questions – Administered to all students at the same time & in the same classroom Written paper

24. Data Analysis Compared percentages of correct responses between the flipped and traditional students – Exam 1 (75 questions) Knowledge questions Application questions – Exam 2 (75 questions) Knowledge questions Application questions – Exam 3 (100 questions) – cumulative Knowledge questions Application questions – Quizzes – Paper

25. Findings Students in the flipped classroom performed similarly to student in the traditional classroom on all measures.

26. Findings: Comparison of Student Outcomes by Teaching Method Student Outcome Teaching MethodMean (SD)p-valueEffect size Exam 1Traditional Flipped 60.2 (3.56) 60.1 (3.39).8740.04 Exam 2Traditional Flipped 57.6 (3.99) 55.9 (4.69).0760.40 Exam 3Traditional Flipped 82.7 (4.01) 83.1 (5.43).6950.09 PaperTraditional Flipped 44.4 (3.02) 45.1 (2.43).2310.27 QuizzesTraditional Flipped 44.0 (2.98) 44.6 (2.72).3400.21

27. What we learned Students learned course content despite: – New approach to learning – Instructors who were novice at flipped classroom methods such as Camtasia and classroom activities Instructors found that teaching in the flipped classroom involved: – More preparation – Online quizzes assessed student knowledge deficits – Greater student involvement in learning – Increased classroom time for application of knowledge

28. What we would do differently Prepare students for the change in teaching methods Use different activities for each classroom versus having similar activities for each Evaluate outcomes other than the traditional examinations such as critical thinking or problem-solving

29. What we would do the same Online quizzes Randomize the groups Flip the classroom

30. New Knowledge Innovative and novel approach to learning in nursing Comparison by randomization Used same method during the entire semester

31. Survey Results: Traditional What did you like most about the traditional classroom approach? Themes: positive interaction; familiarity 13 students- like to ask questions in class 12 students- lecture is familiar and comfortable 12 students- like live lecture and the interaction with classmates 2 students - perceive there was less preparation time

32. Suggested improvements to the traditional approach? Themes: more interaction: reduction in reading/preparation; consistency 17 student - posted slides didn't match what was handed out in class 11 students - needed more practice with NCLEX type questions 5 students- less reading a PowerPoint slides in class 4 students-better organization of content 4 students - too much reading/ too much material three students- more exams

33. What was the biggest challenge to learning in traditional manner? Themes too much material: ineffective class 14 students- too much reading/ too much material 7 students- class is boring; not helpful 6 students - posted notes or slides did not match class notes/slides 3 students - different teaching styles of the professors

34. Have thoughts about traditional classroom changed during semester? If so, why and when? Theme: satisfaction with traditional classroom 29 students- like the traditional classroom 7 students- thoughts changed. After the second test would've liked screen casts lectures, realized variety is important, found out notes different for each group

35. Survey Results: Flipped What did you like most about he flipped classroom approach? Themes: Effective classroom activities; Effective recorded lectures; Autonomy 22 students- liked class activities 18 students- like to recorded lectures 18 students- liked use of the NCLEX questions 8 students- light case studies 1 student - didn't like flipped classroom

36. What improvements could be made to flipped approach? Theme: more effective recorded lectures 14 students- shorter recorded lectures 8 students more effective recorded lectures 5 students requested Word for Word script from recorded lectures 3 students-professors retaught class what was on recorded lectures 1 student too much preparation; explanation for concepts; more frequent tests-

37. What was biggest challenge to learning in flipped manner? Theme: lack of time; lack of immediate clarification 21 students-lacked time to read and/or listen to recorded lectures 6 students-not getting enough immediate clarification while listening to recorded lectures 1 student-notes not aligned with recorded lectures

38. Have thoughts regarding flipped classroom changed during semester? If so, why and when? Theme: satisfaction with flipped classroom 18 students- thoughts did change. They liked flipped classroom better 7 students- no change liked flipped classroom better from start 5 students- liked flipped classroom but then didn't 2 students- thoughts didn't change; didn't like flipped classroom from beginning 2 students-thoughts didn't change- no explanation 3 students- didn't answer question