Presentation is loading. Please wait.

Presentation is loading. Please wait.

Research Paper Critical Analysis Research Paper Critical Analysis 10 ways to look at a research paper systematically for critical analysis.

Similar presentations


Presentation on theme: "Research Paper Critical Analysis Research Paper Critical Analysis 10 ways to look at a research paper systematically for critical analysis."— Presentation transcript:

1 Research Paper Critical Analysis Research Paper Critical Analysis 10 ways to look at a research paper systematically for critical analysis.

2 NUR 599T Human Factors In Healthcare Engineering Dr. Vincent Duffy Research Paper Analysis Submitted by RuthAnn Smolen, RN, MS Ed, HNP, July 19, 2007

3 STUDY CRITIQUE Title: Assessment of Clinical Performance during Simulated Crises Using Both Technical and Behavioral Ratings Authors: Gaba, D., Howard, S., Flanagan B., Smith, B., Fish, K., & Botney, R. STUDY CRITIQUE Title: Assessment of Clinical Performance during Simulated Crises Using Both Technical and Behavioral Ratings Authors: Gaba, D., Howard, S., Flanagan B., Smith, B., Fish, K., & Botney, R.

4 1.Research Ideas or Question (Study and Purpose) Study: “Assessment in clinical performance” Gaba, et.al.) Study: “Assessment in clinical performance” Gaba, et.al.) The purpose is clearly identified in the introduction - The purpose is clearly identified in the introduction - to evaluate ratings of performance (technical and behavioral), and variability of ratings made by multiple independent observers viewing videotapes of simulated crises to evaluate ratings of performance (technical and behavioral), and variability of ratings made by multiple independent observers viewing videotapes of simulated crises The use of human patient simulators (HPS) in health-care training is increasing to gain knowledge, teach technical skills, complete drill training, and human factors training for single or multidisciplinary groups The use of human patient simulators (HPS) in health-care training is increasing to gain knowledge, teach technical skills, complete drill training, and human factors training for single or multidisciplinary groups

5 2. Background Anesthesiology is recognized as the leading medical specialty in addressing patient safety with HPS Anesthesiology is recognized as the leading medical specialty in addressing patient safety with HPS Pioneered development of computer screen and mannequin based interactive patient simulators and training plans ( Schwid, 1987; Small, Wuetz, Simon, Shapiro, Conn, Setnik, 1999). Pioneered development of computer screen and mannequin based interactive patient simulators and training plans ( Schwid, 1987; Small, Wuetz, Simon, Shapiro, Conn, Setnik, 1999). A persistent initiative in graduate education is to find more effective methods to assess specialists’ skills A persistent initiative in graduate education is to find more effective methods to assess specialists’ skills “Techniques are needed to assess anesthesiologists’ performance when responding to critical events” (Gaba, 1998) “Techniques are needed to assess anesthesiologists’ performance when responding to critical events” (Gaba, 1998) Traditionally there has been concern about the potential subjectivity of expert raters. Traditionally there has been concern about the potential subjectivity of expert raters.

6 3. Theoretical Basis/Hypothesis Helmreich, RI Theory underlying CRM training: Psychological issues in flight crew performance and crew coordination (NASA, 1986) in teams Helmreich, RI Theory underlying CRM training: Psychological issues in flight crew performance and crew coordination (NASA, 1986) in teams Behavioral psychologists and learning theories were used originally Behavioral psychologists and learning theories were used originally In anesthesia, patient safety depends on a complex interaction between a skilled practitioner and technology In anesthesia, patient safety depends on a complex interaction between a skilled practitioner and technology A human patient simulator may reduce the risk of learning-curve error in anesthesiologists technical and behavioral clinical performance without subjecting patients to risk A human patient simulator may reduce the risk of learning-curve error in anesthesiologists technical and behavioral clinical performance without subjecting patients to risk

7 4. Literature Search 13 reports published between 1980 & 2000; 4 were designed to investigate validity or reliability of HPS systems; overview of existing evidence of the problem is addressed in follow-up searches of today 13 reports published between 1980 & 2000; 4 were designed to investigate validity or reliability of HPS systems; overview of existing evidence of the problem is addressed in follow-up searches of today In this study the literature review is supportive of purpose and results, current for 1998, and thorough In this study the literature review is supportive of purpose and results, current for 1998, and thorough Summaries of other comparative studies are discussed, relevant to the study and includes strengths and weaknesses Summaries of other comparative studies are discussed, relevant to the study and includes strengths and weaknesses Chopra, et al. (pg. 13) Chopra, et al. (pg. 13) Devitt, et al. (pg. 13) Devitt, et al. (pg. 13) Law and Sherman (pg. 15) Law and Sherman (pg. 15)

8 5. Applicability The aviation industry’s commitment to simulation and similarities between anesthesia and aviation was a key rationale for developing courses in crisis management and using simulation in anesthesia education The aviation industry’s commitment to simulation and similarities between anesthesia and aviation was a key rationale for developing courses in crisis management and using simulation in anesthesia education Generalization among various acute events in simulation teaching and evaluation environments across healthcare disciplines is a reality today Generalization among various acute events in simulation teaching and evaluation environments across healthcare disciplines is a reality today

9 6. Methodologies Raters scored the videotapes of 14 different teams that were managing two scenarios: malignant hyperthermia (MH) and cardiac arrest. Technical performance and crisis management behaviors were rated. Raters scored the videotapes of 14 different teams that were managing two scenarios: malignant hyperthermia (MH) and cardiac arrest. Technical performance and crisis management behaviors were rated.

10 7. Statistical Analyses & Assumptions Using three observers, the study shows good interrater reliability in scoring technical with less agreement between five raters scoring ACRM behaviors. Using three observers, the study shows good interrater reliability in scoring technical achievement with less agreement between five raters scoring ACRM behaviors. The authors analyze the statistics of interrater reliability in depth and note that the scoring system requires further development before it can be used for assess clinical competence for residency/graduation, or board certification The authors analyze the statistics of interrater reliability in depth and note that the scoring system requires further development before it can be used for assess clinical competence for residency/graduation, or board certification

11 8. Presentation of Results What do the results really mean? What do the results really mean? performance varied and correlations among interraters were better for technical markers than for behavioral markers Team performance varied and correlations among interraters were better for technical markers than for behavioral markers

12 9. Conclusions (are they reasonable?) The rating system needs refinement before it can be used to assess clinical competence for residency/graduation, or board certification The rating system needs refinement before it can be used to assess clinical competence for residency/graduation, or board certification This study suggests that simulation can be used to measure more complex skills expected in specialty training This study suggests that simulation can be used to measure more complex skills expected in specialty training Similar to other studies assessing complex technical and behavioral performance multiple encounters are needed to estimate skill effectively and accurately Similar to other studies assessing complex technical and behavioral performance multiple encounters are needed to estimate skill effectively and accurately

13 10. Future Work/Research Direction Studies emphasize the cognitive and social aspects of designing and operating advanced technologies yet to be developed Studies emphasize the cognitive and social aspects of designing and operating advanced technologies yet to be developed The behavioral rating system can be improved – difficulty aggregating a single rating that fluctuated with time The behavioral rating system can be improved – difficulty aggregating a single rating that fluctuated with time Human Factors and ergonomics link quantitative skills to the integrated systems and make sure design of systems act in accordance with requirement of users; includes interactions with technologists Human Factors and ergonomics link quantitative skills to the integrated systems and make sure design of systems act in accordance with requirement of users; includes interactions with technologists Questions: Questions: Is there a need for simulation in assessment of clinical performance? Is there a need for simulation in assessment of clinical performance? Do current simulators match the fidelity of a real-live clinical setting? Are they realistic? Do current simulators match the fidelity of a real-live clinical setting? Are they realistic? What are the potential uses of clinical simulation in medical/healthcare education? What are the potential uses of clinical simulation in medical/healthcare education? What role exists for simulation in healthcare professional training and assessment? What role exists for simulation in healthcare professional training and assessment? Do collaborative opportunities exist? Do collaborative opportunities exist?


Download ppt "Research Paper Critical Analysis Research Paper Critical Analysis 10 ways to look at a research paper systematically for critical analysis."

Similar presentations


Ads by Google