Knowledge Science: Empirical Measures in Knowledge Utilization This area of research has its skeptics: a friend of mine at a conference in Halifax last May scoffed at all this activity as “just communications”. And my response is, well, yes, you’re right…but in the same way that the Internet is more than just 1s and 0s, Knowledge Science is more than just communications. Well, there are difficulties with this area…first, its huge. There is a massive literature out there that is growing every year. There are conceptual difficulties….part of what I’m calling “Knowledge Science” is devoted to producing research that describes/uncovers the conditions for which knowledge is applied, instrumentally, whereas other research studies are devoted only to how the research can be better translated, as in the case where scientific knowledge is translated to fit the appropriate audience: policy knowledge requires knowing the political consequences. For example, research to uncover the “Information literacy skills of Nursing Students” can be considered a type of “knowledge utilization” study whereas the communication of that research to the right people, in the right journals, conferences, websites, administration can be considered a type of “Knowledge Translation” or “Transfer”. This is but the tip of the grand Iceberg of confusion of Knowledge Science. Ultimately though, If research is bringing order or making sense of life then Knowledge Science is bringing order to the huge quantity and growing complexity of research. So, within the context of “Knowledge Science” I want to talk to you about “Knowledge Utilization” providing a snapshot overview. [NEXT SLIDE] Paul J Graham Research Associate & Librarian Department of Sociology University of Saskatchewan December 2008

Presentation Contents Part I: What is Knowledge Science? State of the Art Theory, Methods, Models, Concepts Part II: Knowledge Science Today Nursing Research Research Examples Part III: Problem Based Learning How do we study a real life Problem?

Part I: Knowledge Science: State of the Art

State of the Art: Terminological Tangle Terminology Tangle Knowledge Management, Transfer, Translation, Utilization, Research Transfer, Implementation Research…the list goes on Two Approaches to Knowledge Studies Broad Approach (comprehensive) Searching any possibility, any combination Narrow Approach (Focused) Searching specific terminology, Knowledge Translation There many terms to describe this area of research: I’m using the term “Knowledge Science” to encapsulate the range of activities, processes and ideas that overlap into a genealogy or family of research studies. Here its instructive to consider the work of Ludwig Wittgenstein, Cambridge Philosopher of Language, and one of my heroes (Fortune to sisters; WW I; designs Jet Engine; whistle the Brandenburg concertos)…interestingly he made a good point in the Philosophy of Language on defining words. Try to define the word “Game” to include games like solitaire and Hockey and polo. What we can do is organize games into Family resemblances? Polo and Hockey games with Sticks; solitaire and poker are card games. You can’t find a definition that covers them all, but you can cluster them into family resemblances. And so to with the Terminological Tangle…we need not be disturbed by not having a comprehensive definition, but simply note to cluster them into family resemblances. “Knowledge Transfer” is similar to “Translation” as well as “Diffusion of Innovation” whereas “Application” and “Implementation” have similarity in function of seeing knowledge be utilized instrumentally. This leads to two strategies for understanding the literature: Broad or comprehensive approach where you’re interested in finding all the “Knowledge Science” resources on a topic…for instance, everything on Adult Asthma protocols whether they’re called “Utilization” or “Transfer” or whatever. There is a specific or focused approach where you say, “I want this research to be manageable, specific on the translation of research to policy makers” and so I’ll shape my search towards “Diffusion” “Transfer” “Translation”. The reason being that, even though there is overlap you can still investigate a specific genealogy of a term…it just so happens that “Translation” is more associated with communication, technology transfer and now Health in the use of the word by the CIHR, SHRF and many others. [NEXT SLIDE]

State of the Art: Broad Approach (Comprehensive) Over 1800 publications USA (32%) and Canada (26%) are top producers English articles makes up 94% Top Author: Carole Estabrooks from Nursing Example of searching for many terms: Transfer, translation, Implementation, etc.

State of the Art: Broad Approach (Comprehensive) 1 Management -- R&D, Human resources, Technology networks, Multi-national corporations, cost of information, including information science. 372 2 Nursing – Building Social Capital, Barriers to research utilization, translation to practice, overlapping into evidence based medicine 218 3 Business – Time management, international business, customer knowledge transfer, vertical alliances, absorptive capacity 198 4 Public, Environmental Health – Health technology assessment, teen pregnancy, cancer management 108 5 Operations Research and Management Science – Technology performance, network size and organizational culture, product development process 101 Nursing is second on that broad search. Very impressive.

State of the Art: Narrow Approach (Focused) Over 500 Articles Canada (39%) and US (39%) 99% in English and 1% in German Estabrooks is Top author Example of searching Web of Science for Knowledge Translation and Research Utilization only we see a slightly different picture.

State of the Art: Narrow Approach (Focused) 1 Nursing -- Building Social Capital, Barriers to research utilization, translation to practice, overlapping into evidence based medicine 188 2 Public, Environmental Health - Health technology assessment, teen pregnancy, cancer management 60 3 Health Care Sciences – Mentorship strategy, Research Based Theatre, health care policy making 47 4 Emergency Medicine – Knowledge Translation, adult asthma guidelines, Knowledge Translation systems 33 5 Education, Scientific Disciplines – Stroke rehabilitation programme, Guides to KT, improving performance & evaluation 23 This shows very clearly that terminology does count when you’re investigating these issues. Even the other subjects are primarily health or medical subjects.

State of the Art: Components to Study Subjects Health, Business, Governance, Social Sciences Roles (as intermediaries) Knowledge Brokering, Funding agencies, Implementation Units, Librarians, Database Technicians Approaches or Theory Specific Approaches (Roger’s Diffusion of Innovation model) General Approaches (Social Engineering, Mode 2, Triple Helix) Models Instrumental (Rational) Educational/Enlightenment (Conceptual) Symbolic (Aesthetic) Interactive (Collaborative)

State of the Art: Components to Study Environments to Analyze Policy, Administration, Politics Clinical Application & Technology Theory/Research/Academic (educational) Levels to Analyze Individual (psychological) Group (sociological) Organizational (culture and structure) Methods (General) Bibliometrics; Factor Analysis; Structured Equation Modeling General Surveys, qualitative and quantitative methods Methods (Specific) Devising Special models and ways to interpret Knowledge Science activities. Environment here refers to the surrounding areas of the study. Attitude to research; motivation; intention to share knowledge

State of the Art: General Models Information Utilization Scale Stage of Concern Scale Levels of Use Scale Research Utilization Index Overall Policy Impact Scale Diffusion of Innovations AHRQ Knowledge Transfer Framework

State of the Art: Knowledge System for Clarification Holzner & Marx (1979) Knowledge Application: The Knowledge System in Society The Knowledge System represents the different, but sometimes overlapping, processes in society: Production Management (Organization, Storage & Retrieval) Transfer (or Translation) Application Implementation Evaluation Mobilization (inter-systematic) HAND OUT no. 1

Part II: Knowledge Science Today: Nursing Research

Nursing Research: Setting the Foundation Literacy is the Key The basis of Knowledge Science is the basis of academic activity: critical reading skills, database skills, systematic thinking and synthesis of concepts. Nursing Research: Setting the Foundation Undergraduate Research Programmes Research Mentorship Evidence Based Literacy Number needed to treat; risk assessment; interpreting context of research problem Information Literacy You can’t analyze what you can’t find! Consult with librarians; enroll in skills development Communication Skills & Diplomacy Don’t take your skill set for granted You might consider a Critical Thinking course Vessey & DeMArco, 2008 published a paper on the importance of undergradute programmes Welk (2007) stresses the need for nurses to be able to Paul Graham (at all times) stresses this… I take my skills for granted.

Nursing Research: Bibliometrics Empirical Analysis This is a library science assessment that requires understanding databases; Information Science Skills (Boolean Searching); Content Analysis; Citation Analysis; co-authoring with Librarians. Nursing Research: Bibliometrics Estabrooks et al. (2004) “Mapping the Field” Purpose Map the characteristics of “research utilization” in the nursing research field. Methodology Searching bibliographic databases resulting in 630 articles on research utilization between 1972 and 2001 Used available bibliographic software (freeware from Internet) to organize results Results & Conclusions Models important to Nursing (Iowa Model of Research in Practice, Multidimensional framework, Ottawa Model of Research Use) Top Journals (Journal of Advanced Nursing, Nursing Standard) Important Authors (Cheryl Stetler, Jo-Anne Horsley, Carole Estabrooks) DESCRIPTIVE ANALYSIS We’ll start with reviewing Estabrooks’ Bibliometric work: Bibliometrics is a method for measuring information usage or tracking the effect of authorship. It is a commonly quoted fact at almost all talks that Noam Chomsky gives that he is the most cited academic in the world: The citation of his work puts him in good company: Marx, Lenin, Shakespeare, Aristotle, the Bible, Plato, Freud, Hegel, and Cicero. Estabrooks uses bibliometrics to map the literature, discover who is writing on the topic, how many publications do they have, and what journals are publishing their work. Estabrooks et al have a new general bibliiometric study out titled: “The intellectual structure and substance of the knowledge utilization field” What’s the Relevance? To engage in a bibliometric study is one of the best ways to put your information skills and critical thinking to the test. It is also one of the cheapest ways to start publishing. It is the extension of the skills you have right now and using databases in innovative ways. All of you should be thinking, “How can I turn the subject I’m studying into a bibliometric study?” regardless if its on Knowledge utilization or not. If you were doing research on “Knowledge Utilization” or really, any major project, looking to Bibliometric methods is a good way to at least understand the development of a field. And, just to note: There is another new article titled “bibliometrics, citation indexing and the journals of Nursing” in “Nursing and Health Sciences” [ NEXT SLIDE ]

Nursing Research: Systematic Reviews Empirical Analysis Although using Bibliometrics and Information literacy as the basis, it extends on this basis by needing the nursing knowledge identify specific content for systematic review. Nursing Research: Systematic Reviews Estabrooks et al. (2003) Individual Determinants of research utilization Purpose Review studies that examine individual characteristics & determinants Method Advanced searching of relevant databases Using an inclusion criteria for systematically reviewing specific content Employed a validity Score Results & Conclusions Research was organized into: beliefs and Attitudes, involvement in research activities, information seeking, professional characteristics, education, socio-economic status. Support for a positive association between individual beliefs and attitudes and increased research utilization. Development of a strong interdisciplinary body of work, relevant to nursing is important DESCRIPTIVE ANALYSIS This next study by Estabrooks not only uses bibliometric methods but extends on that skills set to engage in a content review or systematic review. There is still a rigorous search strategy employed looking for resources; however, there are sets of inclusion or exclusion criteria for ignoring some articles. Citations and tracking knowledge is less important that systematically culling out valuable information. For each article, the methodology was to identify if the study includes one or more “Individual determinants of research utilization”; does it measure utilization; specify type of measurement (what instrument was used). They devised a dichotomous score (Met or Not Met) out of 14 possible points; they were interested in studying. Their point system gave weight to objective validity measures over Self-Report. So, self-report gets a lower score. 0-6 were considered weak 7-9 were moderate 10 or more strong All 20 articles that were included shared a weakness: measurement of nurses’ use of research was always by self-report only…I mention this because it poses an interesting contradiction in this study…the authors have chosen to give lower scores to Self-reported measures, but then in the articles themselves state: “the self-reported use of research was measured with acceptable levels of reliability”. What’s the Relevance? Well, we know that Chomsky is the most cited living intellectual…but we don’t know why he’s cited. Doing the review gets us at the content of the articles in question…we know that Chomsky’s work is divided into two major areas: his work as a Linguistics Philosopher revolutionizing the field of language philosophy but we also know he’s a social critic…so why is he being cited. Doing this type of work in Knowledge Utilization means identifying ways that Research Utilization tools are being used via the content. [ NEXT SLIDE ]

Nursing Research: Nursing Models Important Contribution Recommend the creation of a database/website to track, organize and discuss Nursing models in Knowledge Science. Nursing Research: Nursing Models Conduct and Utilization of Research in Nursing Iowa Model of Research in Practice Collaborative research utilization model Stetler/Marram Model for application of research findings in practice Framework for dissemination and utilization of research for healthcare policy and practice Multidimensional Framework BARRIERS to Research Utilization Scale Research Factor Questionnaire DESCRIPTIVE ANALYSIS Here are a few samples of specifically designed research where you might want to apply a survey to a group of clinicians.

Nursing Research: Utilizing a Nursing Model Does the Instrument Fit? The question to ask yourself when you read what instrument is being used is “Does this fit”? Will the model actually measure what the authors want it to measure? Nursing Research: Utilizing a Nursing Model McCloskey (2008) Nurses’ Perceptions of Research Utilization in a Corporate Health Care System Purpose Investigate selected characteristics of nurses based on educational level, years of experience, hospital position that might affect Research Utilization Methods Nurses in five hospitals were surveyed using Research Utilization Questionnaire ANOVA was used to analyze data Results & Conclusion Significant differences (p <.001) were found in perceived use of research, attitude toward research, availability of research resources, or perceived support for research activities based on education and organizational position. Fit of instrument: What is really great in this article is that they actually have an assessment of the instrument allowing you to follow up on the information. If there is no information on the fit of the instrument then, if you are unaware of that tool, that’s a sign you’d better look it up, find out if that tool has been valididated…which leads me to.

Nursing Research: Validating a Nursing Model Reflect on Knowledge Process Communities of Practice are important to this model; it is a concept that is overlapping with a large “Knowledge Management” literature, another important part of Knowledge Science. Nursing Research: Validating a Nursing Model Tolson et al. (2008) “Achieving evidence-based nursing practice” Purpose Determine the impact of the Caledonian Development Model (version 2), designed to promote evidence based practice Participants engage in Communities of Practice, focus on a specific goal, and learn how to use and participate in a “virtual college” Method 24 nurses and 18 practice sites formed 3 Communities of Practice. Pre- and post intervention Revised Nursing Work Index; focus groups SPSS and Nvivo used for analysis Results The model performed well as an agent of knowledge translation supporting the implementation of values-based evidence-based care guidance within the context of nursing older people. Purpose The Caledonian Model is the product of a longitudinal action research study (2006) which sought to develop, in partnership with practicing nurses, a sustainable approach to attaining evidence based nursing care of older people. The main component that binds the study together is using the Community of Practice model, developed and originating in the Knowledge Management literature (Wenger). There is an emphasis here on Face to Face meetings alluding to Tacit Knowledge exchange. Use of Cops, Best practice statements, and a Virtual college In this case, each Community of Practice selects a Best Practice statement as a focus for developmental activities (Preventing Depression, Nutrition and physical activity) and learned over 6 months how to use the “virtual college”. Method of Evaluation The purpose is to test the impact or evaluate the model…so in the method that’s why another tool or instrument is used to measure effectiveness. They’re engaged in the process for the Caledonia Model…and then asking, Okay, fill out this survey to give us feedback on importance. Data Audits Facilities Audits: Patient Audits:

Nursing Research: Evaluation or Testing Empirical Analysis We are still interested in the individual determinants, but we are building on our previous examples: you need to know more quantitative analysis, structural equal modeling & LISREL Nursing Research: Evaluation or Testing Estabrooks (1999) Modeling the Individual Determinants of Research Utilization Purpose This study’s objectives were to develop and test a series of structural equation models that included individual factors believed to influence research utilization Method Using Instrumental, Conceptual & Persuasive research utilization Using a 1996 Nurses Survey Model fit was assessed by examining chi-square, the adjusted goodness of fit index Scaling and reliability in LISREL Results Some individual factors: Activism, Affiliation, attitude, Autonomy, Awareness of policy, Belief suspension, Cosmopoliteness, Dogmatism, Educational level, Problem solving ability, Theoretical orientation, Time spent reading professional journals, Trust DESCRIPTIVE ANALYSIS This study is an example that puts some of our conceptual models to the test: These are not the constructed research quesionnaires but our previous conceptual definitions for types of Knowledge Utilization: Instrumental, Conceptual, Symbolic; there is also measuring of individual factors. What’s the Relevance? We’re introduced now to the concept of using Structural Equation modeling for testing, not developing, Utilization research. SEM is used for testing and LISREL is a software package for doing SEM. The person who uses this in our department is Dr. Liz Quinlan; she is sharp with her analysis. “We must conceptualize and operationalize research utilization in ways that reflect the idea that nurses’ personal assessments of instrumental, conceptual, and persuasive research utilization contribute to their assessment of overall research utilization.” Cosmopolitness: Degree to which a nurse is oriented outside the immediate work sphere.

Nursing Research Clinical Realm Reflect on Types of Models The author calls the work a “description” of research activities, similar to the Educational or Conceptual Model of Knowledge Utilization Nursing Research Clinical Realm Oh, E.G. (2008) Research activities and perceptions of barriers to research utilization among critical care nurses in Korea Purpose Analyze data from Nurses and Nurse Managers working in critical care units to describe research activities and to identify barriers Method Data from 63 critical care nurses describing research activity such as reading journals / week, additional nursing courses, etc. Barriers to Research Utilization Scale (Funk, 1991) used. Results Critical care nurses share a strong sense of value for the contribution that research makes to improving their practice, bus also perceived barriers in the lack of administrative support The nurse does not see the value of research for practice. The nurse feels the benefits of changing practice will be minimal. Administration will not allow implementation. • Physicians will not cooperate with implementation. Results Reading frequency and research participation rates were lower compared to previous studies: results of low research activities indicate that evidence-based nursing is a relatively new and emerging concept in Korea.

Nursing Research: Nursing Culture Mixed Methods We can note the role of qualitative data as the preliminary stage in forming Knowledge Utilization theory or method. Nursing Research: Nursing Culture Watson et al. (2005) Exploratory Factor Analysis of the research and development culture index among qualified Nurses Purpose Factor analysis of a rating instrument for assessing the strength of organizational Research and Development (R&D) Culture Methods Qualitative research is the basis for creating the formal index R&D Culture Index used as means of appraising R&D culture Results Both Individual and Organizational Factors are important and distinctive. Three components must be developed: Skill base of practitioners, infrastructure of organizations, and reflective development nature of the immediate work environment Focused on practice development and research capacity and culture Creating a climate to fulfill potential Creating an environment that fosters and approves of research oriented activities.

Nursing Research: Organizational Measures Organizational Readiness The Canadian Health Services Research Foundation developed a “readiness” assessment guide for analyzing the organization’s preparedness Nursing Research: Organizational Measures Fink, Thompson, Bonnes (2005) Overcoming Barriers and Promoting the Use of Research in Practice Purpose Identify Nurses changes in attitudes toward research utilization and the organization’s environment pre-implementation and post-implementation. Method Descriptive, cross-sectional, pre-survey, post-survey design to identify inpatient nurses’ personal and professional characteristics, perceptions of organizational culture and the effectiveness of organizational strategies. Results & Conclusion Research activities most participated: Shared research findings with peers (70%), attended conference (67%) Evidence based council meetings (55%); participant in data collection (n = 123), attending Journal club (n = 105) Barriers to Research included: Nurse had no authority to change practice, aware of research, administration not allowing implementation, nurse is isolated from colleagues, access to research, no time on the job to read research. AIMS identify nurses’ attitudes and perceptions about organizational culture and research utilization, identify perceived barriers and facilitators to nurses’s use of research in practice and 3) determine which factors are correlated with research utilization. Instruments: BARRIERS to Research Utilization Scale and Research Factor Questionnaire. In the particular study, they claim to have supported/validated the Rogers Diffusion of Innovations theory constructs. So we see Research Utilization studies fulfilling a confirmatory role in justifying theory.

Nursing Research: Qualitative Analysis Using Qualitative Software Nvivo 7/8 is now available at the university; you can seek out training from the Qualitative Research Centre. The author provides Nvivo sessions. Nursing Research: Qualitative Analysis Thompson et al (2008) The relationship between busyness and research utilization Purpose Explore the concept of busyness in nursing and to understand the relationship between busyness and nurses’ research utilization better Method Conducted a secondary analysis of data collected in a large ethnographic multiple case study examining the use of research by Canadian nurses in the context of adult and pediatric pain management Results & Conclusions Lack of time or busyness refers to the mental time and energy required to use research and the culture of busyness in nursing rather than physical time required to perform routine tasks.

Nursing Research: Research Environment The Reality of Implementation The need for education and attitudinal changes are real; you will need to know how to communicate new ideas, changes and be able to argue for their implementation. Nursing Research: Research Environment Ax, Susanne (2001). Nursing Students’ Perceptions of Research: Usefulness, Implementation and Training Purpose Investigate Nursing Research perceptions of usefulness, implementation, interest and personal involvement Investigate Nursing Training in structure and design of classes, reading materials, personal study and stress Methodology One to one Interviews of 13 Nursing Students Findings Improving Practice by testing assumptions of current practice Many students voiced difficulties voicing suggestions for change Feeling of a lack of understanding, a disconnect to application The findings were interesting: --Nursing Students felt they had a role in testing and improving research application by being involved in studies. “I think its very important that nurses can do their own research when required. . . I think it gives them a better understanding, say for drug trials and that. They can use their own input, can’t they? They can put their own ideas forward if they’re interested in a certain subject, or a specialty that they work in , they can put their input into it…to help better deliver the care or something like that.” There was reported difficulty in suggestions for change “I found that research was relevant to practice, although you don’t like saying that when your’re out on the wards. If you say ‘Oh, there’s some research which says this’, you can hear a wave of sighs and they say “Oh no, there’s a student on the ward’. “Don’t bother with that, get your gloves on and your hands in there.” Lack of Understanding was voiced as a feeling of disconnect between course work and “real work” nursing. The general sentiment here was that although research was important, there was difficult seeing how it will occur in their nursing placement: “I don’t really understand how we will use it when we are nursing”. And there were other components to this study as well: Students indicated that the course work had helped them learn how to read research papers. Students felt that skills they learned would help them in everyday life. Many had thought about being involved in research projects after a few years of working life.

Part III: Problem Based Learning Plan your own Knowledge Science Project Let’s Apply what we’ve learned…

Possible Scenario: Descriptive Assessment Knowledge Management Identify Knowledge Brokers (800 million rural poor) Professionals Local Authorities Trusted persons Cultural Component/Indexes Mapping the flow of knowledge Diffusion of Innovation Conceptual/Educational Model Learning about the System Level of Analysis Policy AND Practice level (Government)