Presentation is loading. Please wait.

Presentation is loading. Please wait.

Guide to CAOT Abstract Review

Similar presentations


Presentation on theme: "Guide to CAOT Abstract Review"— Presentation transcript:

1 Guide to CAOT Abstract Review
Prepared by the CAOT Conference Scientific Program Committee The CAOT Scientific Program Committee members are: Mary Manojlovich (chair) Jacquie Ripat (past chair and committee member) Susan Nesbitt Noemi Cantin Melissa Nance Melissa Croskery Joanie Pelletier Krista wade Michelle Ryan Joanne Hanlon The committee would like to thank Fern Swedlove, pat committee member and editor of OT NOW for her expertise and assistance in preparing this presentation.

2 Introduction: Overview
This Webinar is intended to provide information, guidance and support to members of the CAOT Abstract Review Board. The presentation will review the rationale, process and outcomes of the abstract review process, and provide examples of rated abstracts. Abstract writers may also find this information useful. The CAOT Scientific Program Committee surveyed the members of the 2007 Abstract Review Board for suggestions about information that might be helpful in assisting future reviewers. Much of the information in this presentation reflects their input. We acknowledge their contributions and appreciate their enthusiasm for this project. © Canadian Association of Occupational Therapists

3 © Canadian Association of Occupational Therapists
Introduction The objectives of this Webinar are: To improve consistency of ratings among reviewers. To suggest factors for consideration when rating abstracts. To suggest an approach to the review process. To provide examples of rated abstracts. To provide examples of helpful comments. To provide additional links and resources. To enhance the experience for the reviewer. The CAOT website provides resources which are useful to the abstract reviewer. These include the Abstract Submission Guidelines and the Abstract Review Rating Form. The information provided in this presentation is meant to complement these resources by providing more detail and addressing common concerns encountered by reviewers when reviewing abstracts for the CAOT conference scientific program. © Canadian Association of Occupational Therapists

4 © Canadian Association of Occupational Therapists
Introduction “I think the most important consideration when reviewing an abstract is to ask myself if I would want to use a half hour or hour of my finite life to listen to that presentation,” said one reviewer. Quote from the 2007 Abstract Review Board Survey. © Canadian Association of Occupational Therapists

5 Organizational Chart Conference Steering Committee
Conference Host Committee Conference Scientific Program Committee The Conference Steering Committee has representatives from the Conference Host Committee, the Conference Scientific Committee and the CAOT National Office. This committee oversees the overall conference planning process. The Host Committee organizes the local social events and volunteers and has a strong voice in decisions regarding the conference theme, keynote speaker, and local promotion of the conference. The Scientific Program Committee plans the scientific program and makes the final decisions about which abstracts are accepted. The Abstract Review Board is made up of CAOT members who volunteer to review abstracts for the committee. Each Review Board member reviews up to 20 abstracts using the Abstract Review Rating Form and submits their ratings to the Scientific Program committee. Abstract Review Board

6 Outcomes of review process
A scientific program that….. Is of high quality and excites conference attendees Reflects innovation and diversity of occupational therapy research, practice and professional issues Reflects a balance between research, practice, professional issues and education Reflects a balance between child/adolescent, adult and older adult physical health and psychosocial health Provides constructive and supportive feedback to abstract authors The process of abstract review is critical to a scientific program of high quality. It can also play a role in building capacity for research by providing feedback to abstract writers that will assist them to improve their acceptance rate for future conferences. © Canadian Association of Occupational Therapists

7 Overview of abstract review process
Submitted abstract matched anonymously to three volunteer reviewers based on self-declared knowledge of reviewer Abstracts reviewed using CAOT Abstract Review Rating Form Reviews submitted to Conference Scientific Program Committee Abstracts assigned a rating of “accepted”, “waitlisted” and “not accepted” by committee based on ratings, comments and program balance Presentation assigned a timeslot in the program CAOT members are invited to volunteer to become an abstract reviewer each year through completion of a volunteer application form. These volunteers form the Abstract Review Board. Each volunteer completes a questionnaire that provides information for matching their expertise to the abstracts submitted. The mean score of accepted abstracts in 2006 was 39.8/ abstracts were submitted, and 200 accepted. An attempt was made to ensure that 50% of abstracts in each category were accepted. For example, if 30 abstracts were submitted in child physical health, 15 is the goal for number of abstracts accepted. © Canadian Association of Occupational Therapists

8 Reviewer Self Reflection
Do I have sufficient knowledge in the abstract content and methodology to provide a fair review? Do I base my ratings on a consideration of all aspects of the abstract review form or do I focus more on grammar and writing style vs. content? For more information on qualitative and quantitative research and research design options and statistics, refer to If you still feel unable to fairly evaluate the abstract, please return to as soon as possible, so the abstract can be reassigned. © Canadian Association of Occupational Therapists

9 Considerations for research abstracts
Is the research quantitative or qualitative? Do I understand the research design options? Do I understand the methodology outlined in the abstract? Is it appropriate for the question? Do I understand the statistical terms used? Are the statistics appropriate to the research question? For more information on qualitative and quantitative research and research design options and statistics, refer to © Canadian Association of Occupational Therapists

10 Practice, Professional Issues and Education Abstracts
Does the abstract provide new information or approach the topic in a novel way? Does the abstract address issues that are important to the profession? Is the information based on a theoretical approach? Will the content change practice? Conference attendees include the full range of practitioners in our field. There are researchers, educators, managers, practice leaders, and clinicians, with clinicians typically making up the majority of the audience. It is helpful to keep this in mind when reviewing abstracts. Non-research abstracts should offer something to the conference attendee that will be new and will advance practice in some way. © Canadian Association of Occupational Therapists

11 Approaches to the Abstract Review
No right or wrong approach Most reviewers read all abstracts first, before rating to get a “general feel” for the overall range Some reviewers then rank order the abstracts or create a grading rubric Reviewers then rate each abstract individually using the rating form These approaches were suggested by the 2007 Abstract Review Board. © Canadian Association of Occupational Therapists

12 © Canadian Association of Occupational Therapists
Rating the Abstract I read them all through first, thinking initially do I get a picture of what they are talking about, does it made sense, some are so clear yet others I have to read several times to get at what they are saying (and that tells me lots).” Quote from the 2007 Abstract Review Board Survey. © Canadian Association of Occupational Therapists

13 © Canadian Association of Occupational Therapists
Rating the Abstract “The strategies that I use to do reviews is to honour the criteria that is provided, seek clarification if I am uncertain about information within or missing from the criteria and add to the criteria as I go through the abstracts to ensure that I am using the same criteria across the board.” Quote from the 2007 Abstract Review Board Survey. © Canadian Association of Occupational Therapists

14 © Canadian Association of Occupational Therapists
Rating the Abstract “I jot notes on each abstract and give each a quick mark. I then set aside the abstracts that I reacted to most strongly - both positively and negatively…in a day or two I re-read and re-mark each abstract in a more careful manner….I find that a bit of time for reflection helps me determine what engendered that strong positive or negative reaction on initial reading and whether that response was valid. These abstracts often have a more significant change in mark on second reading.” Quote from the 2007 Abstract Review Board Survey. © Canadian Association of Occupational Therapists

15 Approaches to the Abstract Review
Some reviewers comment before scoring; others after Reviewers check their consistency in a variety of ways – some rate the abstracts at 2 sittings independently, then compare scores Others rank order after scoring, then compare scores to see if they are consistent with the ranking Some reviewers review highly and poorly rated abstracts again to ensure fairness © Canadian Association of Occupational Therapists

16 Rating the Abstract: Quality of the Presentation Content
Headings are used as indicated on the form Incomplete or absent heading and content receives a score of 1 Introduction or rationale must provide a clear background to the rest of the abstract, and should be reinforced in the conclusion Objectives must outline the content or expectations of either the project (generally appropriate for research, practice and education topics) or the presentation (may be more appropriate for professional issue topics) © Canadian Association of Occupational Therapists

17 Rating the Abstract: Quality of the Presentation Content
Methods or approach must provide a clear description of the methodology used, and it must be appropriate to the objectives and rationale of the presentation or project Results or practice implications must indicate clearly the findings of the project, and they must be consistent with the methodology and objectives © Canadian Association of Occupational Therapists

18 Rating the Abstract: Quality of the Presentation Content
Conclusions must be consistent with the introduction or rationale and objectives, so that the information is complete Continually refer back to the system outlined on the form while rating (unacceptable means information is incomplete or absent) © Canadian Association of Occupational Therapists

19 Rating the Abstract: Educational Value
Reviewer should consider potential audience for the presentation – what would be of interest to them? Is the content relevant to an occupation-based approach? Does it bring a perspective that is relevant to current practice? Does the presentation have the possibility of changing current practice? Does it add significantly to the current body of work in this area? © Canadian Association of Occupational Therapists

20 Rating the Abstract: Educational Value
Does the presentation reflect the theme of the conference? Is the information novel in some way? Is the approach or methodology new or different from known approaches? Do the results provide support for a new approach or for changing an accepted approach? Are the ideas presented provocative? © Canadian Association of Occupational Therapists

21 Rating the Abstract: Quality of Written Abstract
It is important to consider grammar and writing style in this section only, and not let poor grammar influence all ratings; some readers rate this section first and rate on first impressions Check here for biases in preferred writing styles; try to be objective References should be confined to 1 to 3 and essential for appropriate recognition of authorship in the abstract Abstracts should be clear on first reading; repeated readings for clarity indicate lower readability © Canadian Association of Occupational Therapists

22 Considerations for Reviewer Comments
“I always comment. I usually discuss the potential audience…If I think another format would be better or if I have some advice re presentation I include that. I also comment on any lack of clarity (often by asking a question) or wording problems.” Quote from the 2007 Abstract Review Board survey. © Canadian Association of Occupational Therapists

23 Considerations for Reviewer Comments
“I try to put something on each abstract as I know how helpful it is to have some feedback. Positive feedback is very motivating! Constructive feedback, stated in a helpful manner is too.” Quote from the 2007 Abstract Review Board survey. © Canadian Association of Occupational Therapists

24 Considerations for Reviewer Comments
“I recognize that some topics appeal only to a small audience, but they are keenly interested…I try to balance breadth of interest vs. depth of interest.” Quote from the 2007 Abstract review Board Survey. © Canadian Association of Occupational Therapists

25 Considerations for Reviewer Comments
Consider the tone of the comments – are they constructive and helpful? Do they suggest a better approach? Low scores should be supported by a comment Comments should be specific, so the reader can learn how to improve A different presentation format (such as a poster format) may be suggested © Canadian Association of Occupational Therapists

26 Examples of helpful comments
Sandwiching the comments (i.e. begin and end with a positive statement) “The presentation topic is important for advancing the profession” Asking questions for clarification “How do the gaps in the literature support the need for this study?” “There appeared to be several objectives for this study. Can you clarify the specific objective that the abstract pertains to?” “Analysis of the data is not specified. How were the results analyzed?” “How were improvements determined?” “How does the topic link to occupational therapy practice?” © Canadian Association of Occupational Therapists

27 Examples of helpful comments (cont’d)
Statements on linking or extent of information provided “The rationale does not appear clearly linked to the study objectives and methods.” “The methods section does not provide enough information to determine whether the project will meet its objectives. More information on the methods used is necessary.” Comments that are specific to improving a particular aspect of writing “Ensure consistency of tense use.” © Canadian Association of Occupational Therapists

28 Examples of unhelpful comments
Non-specific comments “This a very intriguing study, however, I found the methods, results and conclusions sections confusing.” “Abstract is vague” Overly specific comments “I would add ‘an’ in front of ‘information’ in the methods section” Comments that judge the value of the work “I’m not sure whether this project will contribute to occupational therapy practice” Comments that use a judgmental tone or imply burden on the reviewer “I find the writing very confusing and had to re-read it several times to understand what the author was getting at” © Canadian Association of Occupational Therapists

29 Abstract Example: Well-written Abstract
Introduction: Partners of individuals with dementia often experience challenges with their ability to manage the daily issues that arise with being the primary caregiver. Personal stress, increased physical demands, personality changes in their partner and dealing with their own aging are common issues that partners may face. Occupational therapists are often consulted to work with the individual with dementia, however it is often the partner that is the primary recipient of services. Little is known about how male partners are able to cope with the daily challenges of being the primary caregiver. Objectives: The purpose of this phenomenological study was to explore the lived experience of the male partners of individuals with dementia to develop an understanding of the strategies they use to support their partner at home. © Canadian Association of Occupational Therapists

30 Abstract Example: Well-written Abstract (cont’d)
Methods: Using purposive sampling, male partners of individuals with dementia were recruited for this study. Participants were individually interviewed using a semi-structured questionnaire. Interviews were audio-taped, transcribed verbatim and reviewed to develop a coding scheme and definitions. Two researchers conducted line-by-line review of the transcripts to identify themes and sub-themes that emerged from the data. Additional notes were made throughout the analysis to document further ideas and assumptions. The summary of themes was sent to participants for member checking. © Canadian Association of Occupational Therapists

31 Abstract Example: Well-written Abstract (cont’d)
Results: Data saturation was achieved after analysis of the data from twelve participants. Three sets of primary coping strategies emerged from the data and were confirmed by participants. Longing for the Past, Day-to-day Survival, and A Predictable and Unwanted Future were themes that encapsulated the finding of this study. Conclusion: The results of this study will assist occupational therapists to understand the coping strategies that the male caregivers of individuals with dementia use to maintain their partners at home. Identifying ways to support these primary caregivers is an important role for occupational therapists as our population ages. © Canadian Association of Occupational Therapists

32 Quality of Presentation Content (23/25 points)
Introduction (5/5) The author provides justification for the study. Objectives (5/5) Objective of project is specific Methods (4/5) Methods used to conduct the study are clear. Partners are not necessarily the same as caregivers, although they appear to be synonymous in this study. This could be clarified. Results (5/5) Results are identified Conclusion (4/5) Some repetition in the conclusion that doesn’t add to the abstract. © Canadian Association of Occupational Therapists

33 Educational Value (13/15 points)
Interest and appeal to an occupational therapy audience (i.e. needs to be heard, describes historical perspective, current trends or new ideas) 4/5 May have a limited audience as it addresses a very specific population and area of practice. Important contribution to practice, research, theory or knowledge 5/5 An interesting topic and I am eager to hear more about the results and implications for occupational therapists working in this area, in particular I look forward to a discussion and more details on the strategies. © Canadian Association of Occupational Therapists

34 Educational Value (13/15 points) con’t
Novel or innovative contribution, relevant to the conference theme 4/5 Although not a novel issue, the author suggests that limited work has been done to explore this issue and thus anticipate that the contribution is novel. Author does not clearly link the topic to conference theme. © Canadian Association of Occupational Therapists

35 Quality of Written Abstract (9/10 points)
Self-contained (i.e., should not include abbreviations, acronyms, quotes or extensive reference citations) and concise/specific (i.e., each sentence is maximally informative, especially the lead sentence) 5/5 Coherent and readable (i.e., written in logical sequence, use of clear vigorous prose, use of the active not passive voice, avoids use of personal pronouns) 4/5 Abstract is clearly worded. Uses passive voice at times. © Canadian Association of Occupational Therapists

36 Abstract Example: Poorly written abstract
Introduction: With the increasing number of older adults “aging in place”, older adults are at risk for falls in their home. Person and environment factors can contribute to a situation that may result in injury. The result of falling can be detrimental to one’s health and quality of life as it may result in prolonged hospitalization. Objectives: To develop an in-home falls program to reduce the incidence of falls in the well elderly. Methods: Participants met with an occupational therapist to learn about home safety techniques on a weekly basis over a three month period. They used the Falls-Reduction Inventory to educate seniors on possible home hazards. © Canadian Association of Occupational Therapists

37 Abstract Example: Poorly written abstract con’t
Results: Twelve seniors participated in the project. Participants completed an initial evaluation of hazards in their home and then worked with the OT to develop solutions to reduce the risk of falls. A total of 88 home hazards were identified by the participants. Interviews with the older adult participants suggested that they felt more capable of identifying risks in their home environment. Conclusion: The results of this project support the involvement of occupational therapists in working with seniors on falls prevention in their home. © Canadian Association of Occupational Therapists

38 Quality of Presentation Content (13/25 points)
Introduction (3/5) The abstract highlights the issue of falls in older adults. However, clarification of the problem leading to the need to do this project would be helpful. Further background information on previous work and the gaps in that work would also assist to justify the need for this project. Objectives (4/5) The objective stated is clear, however, is this the true objective of the study? By reading the methods, it appears that the project was not about program development. © Canadian Association of Occupational Therapists

39 Quality of Presentation Content (13/25 points)(cont’d)
Methods (2/5) The methods used to address the issue are not clearly linked to the objectives of the project. For example, how were interviews used and analyzed? More specific information on the methods used would be helpful for understanding how data was collected. Could you describe the “Falls-Reduction Inventory” ? Further information on this inventory would assist the reader to understand how it was used in this project. Results (2/5) The statement “Participants completed an initial evaluation of hazards in their home and then worked with the OT to develop solutions to reduce the risk of falls” is better suited to the methods section as it addresses “what was done” in the study. © Canadian Association of Occupational Therapists

40 Quality of Presentation Content (13/25 points)(cont’d)
Results con’t Results of the data collected are described, however, it is difficult to interpret the results as they do not clearly link to the methods section. Clearly linking methods and results would help the reader make this connection. Conclusion (2/5) The final statement of conclusion is not entirely consistent with objectives of the study, this makes it difficult to determine whether the project was successful. It is not clear how the results will contribute to the knowledge base of occupational therapists’ understanding of this area of practice; a strong statement about this contribution would strengthen the abstract. © Canadian Association of Occupational Therapists

41 Educational Value (8/15 points)
Interest and appeal to an occupational therapy audience (i.e. needs to be heard, describes historical perspective) (5/5) This is a topic of interest to occupational therapy clinicians and the topic area would have broad appeal to the conference delegates as it addresses a common area of OT practice. Important contribution to practice, research, theory or knowledge (1/5) Unfortunately, limited information and lack of clarity on the methods used make it difficult to determine how this project can contribute to furthering OTs understanding of the problem. Novel or innovative contribution, relevant to the conference theme (2/5) The author could clearly articulate the novel contribution of this project to the topic area; further development of a justification for the project in the introduction section would address this. © Canadian Association of Occupational Therapists

42 Quality of Written Abstract (4/10 points)
Self-contained (i.e., should not include abbreviations, acronyms, quotes or extensive reference citations) and concise/specific (i.e., each sentence is maximally informative, especially the lead sentence) (2/5) The first sentence could provide more information as to how it links to the project; reversing the clauses would strengthen that sentence. The term “aging in place” should be elaborated on, or not used rather than put in direct quotes. Some sentences require further development (e.g. They used the Falls-Reduction Inventory to educate seniors on possible home hazards). OT is used as an acronym without defining it first. Each sentence should stand on its own as a complete sentence (e.g. see objectives). © Canadian Association of Occupational Therapists

43 Quality of Written Abstract (4/10 points) (cont’d)
Coherent and readable (i.e., written in logical sequence, use of clear vigorous prose, use of the active not passive voice, avoids use of personal pronouns) (2/5) The use of various terms to describe the participants (seniors, older adults, elderly) is somewhat confusing; I would suggest that the author use consistency in terms and ensure that person-first language is used. It would be beneficial to review the abstract for grammatical accuracy and clarity of writing. Use of personal pronouns is discouraged. © Canadian Association of Occupational Therapists

44 © Canadian Association of Occupational Therapists
Poster Presentation Useful to present lots of data and other information that is fairly self-explanatory, or for presentation of a few key ideas The reviewer may want to suggest that the writer consider another format for their presentation. The 3 possibilities are: poster, paper, and extended session. © Canadian Association of Occupational Therapists

45 © Canadian Association of Occupational Therapists
Paper Presentation Information must be appropriate to a 20 minute time frame Presentation should present ideas that require verbal explanation and can’t be presented effectively in a written format © Canadian Association of Occupational Therapists

46 © Canadian Association of Occupational Therapists
Extended Session The extended session is appropriate for topics that require participation and engagement by the audience The author must indicate how this type of session would be facilitated It is not an appropriate format for presentations that appear to be extended papers only (that have no indication of a delegate learning component) © Canadian Association of Occupational Therapists

47 Reviewing Dilemmas No results
Abstracts on works-in-progress are accepted Writers should provide information on preliminary results or preliminary trends if available If no results are available, authors should discuss practice implications “the results of this survey will inform occupational therapists on the most relevant assessments used by therapists working with…” This is a controversial area. Some reviewers believe that abstracts should not be accepted unless results are known. However, the Scientific Program Committee believe that it is appropriate to submit an abstract even though the project is still in progress, if the reviewer is confident that he results will be available by conference. © Canadian Association of Occupational Therapists

48 Reviewing Dilemmas Well written but poor content
Try to address the potential significance of this work Has little work been done or reported on in this area? Is this an emerging area of practice? Check yourself for reviewer bias – is this a content/practice area that you are familiar with? Ensure that the educational value section marks reflect your comments and perspective © Canadian Association of Occupational Therapists

49 Reviewing Dilemmas Poorly written but interesting content
Suggest that reviewers evaluate the quality of writing first, then re-read the abstract to rate the quality of content © Canadian Association of Occupational Therapists

50 Reviewing Dilemmas Unfamiliar methodology or terminology
Highlight unfamiliar terms or methods in the first review and look them up (see resources and links page) Focus more on substantial methodological issues rather than details as these are difficult to assess in a 250 word abstract Balance judging scientific merit with the author’s ability to convey their results and interpretation If you feel unqualified to review a particular abstract, let Gina Meacoe know and ask that the abstract be re-assigned © Canadian Association of Occupational Therapists

51 Helpful links and resources
Abstract Submission Guidelines, Tips for Preparing Abstracts, provides useful information on assessing the methodological evidence in published papers. This will also be applicable in this context. McMaster University, School of Rehabilitation Science – Evidence-based Rehabilitation provides guidelines to appraising quantitative and qualitative literature. © Canadian Association of Occupational Therapists

52 © Canadian Association of Occupational Therapists
Feedback Have you found this presentation helpful? Is there too much information/not enough? Do you have suggestions to improve this presentation? If so, please Thanks for your input! © Canadian Association of Occupational Therapists


Download ppt "Guide to CAOT Abstract Review"

Similar presentations


Ads by Google