Presentation is loading. Please wait.

Presentation is loading. Please wait.

Science of Team Science A Concept Mapping Project William M. Trochim Presentation to First Annual Conference on the Science of Team Science Northwestern.

Similar presentations


Presentation on theme: "Science of Team Science A Concept Mapping Project William M. Trochim Presentation to First Annual Conference on the Science of Team Science Northwestern."— Presentation transcript:

1 Science of Team Science A Concept Mapping Project William M. Trochim Presentation to First Annual Conference on the Science of Team Science Northwestern University, Chicago, IL Thursday, April 22, 2010 This presentation contains draft results from studies that are still in progress. It may not be reproduced or distributed without written permission from the author.

2 Project Goals Develop a “roadmap” of a comprehensive research agenda for the science of team science –Help orient participants during the conference –Provide a framework that can help inform the field Use a team science approach to mapping team science –Structured concept mapping Integrative mixed methods approach Integrates group process with statistical analysis Utilizes web technology for distributed participation Provides a rigorous visually interpretable result

3 Define the Focus “One topic that should be part of a comprehensive research agenda for the science of team science is…” Develop a focus

4 Identify Participants “One topic that should be part of a comprehensive research agenda for the science of team science is…” All conference invitees were asked to Brainstorm & Rate - 63 Rated 15 steering committee members sorted Identify the participants Develop a focus

5 Brainstorm Outcomes 95 240 89using publication and bibliometric data (e.g., citation rates, impact factors) to assess team science 69importance of developing multi-method strategies to assess processes and outcomes of team science 2how to evaluate success of team science-based research centers 65measuring effectiveness of team science on multiple levels: individual team, impact of research, effectiveness of team science funding programs, etc. Generate Ideas Identify the participants Develop a focus

6 Organize Outcomes Importance Rate the ideas Sort the ideas Structure Ideas Generate Ideas Identify the participants Develop a focus

7 Compute Maps 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 7 8 3 41 2 5 6 Compute Maps Aggregate sorts Multidimensional scaling Hierarchical cluster analysis Structure Ideas Generate Ideas Identify the participants Develop a focus

8 Interpret Maps 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 7 8 3 41 2 5 6 Interpret Maps Compute Maps Structure Ideas Generate Ideas Identify the participants Develop a focus

9 Use Maps Interpret Maps Compute Maps Structure Ideas Generate Ideas Identify the participants Develop a focus Utilize Maps Provide initial roadmap for conference and the field r =.51 Area 1Area 2 4.22 3.47 4.4 3.56 Community & Consumer Views Management Information Services Regionalization Technology Financing Mission & Ideology Technology Community & Consumer Views Information Services Management Regionalization Financing Mission & Ideology

10 Who participated?

11 Gender

12 Education

13 Academic Discipline

14 Employment

15 Team Science Focus

16 Experience with Team Science

17 Professional Experience

18 How did we obtain the results?

19 This initial map shows all the potential outcomes in relation to one another 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95

20 6heterogeneity of team membership Each point represents one of the brainstormed outcomes 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95

21 91status differences and power dynamics within the team 44team member interchangeability 6heterogeneity of team membership Conceptually similar outcomes are in close proximity 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95

22 81co-authorship and multi-PI authorship in team science 90use of collaborative computerized tools to support and enhance team science 58evaluating and learning from successful teams 91status differences and power dynamics within the team 44team member interchangeability 6heterogeneity of team membership Conceptually different outcomes are further apart 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95

23 The outcomes are organized into clusters 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 This map shows each of 95 statements grouped into seven clusters by hierarchical cluster analysis.

24 The Map Results

25 Measurement & Evaluation of Team Science 89using publication and bibliometric data (e.g., citation rates, impact factors) to assess team science 69importance of developing multi-method strategies to assess processes and outcomes of team science 2how to evaluate success of team science-based research centers 65measuring effectiveness of team science on multiple levels: individual team, impact of research, effectiveness of team science funding programs, etc. 8measurement of key constructs (e.g., collaboration, disciplinarity, team effectiveness, personal/behavioral characteristics, team processes, readiness; synergy, productivity, shared knowledge) 46best approach(es) to assessing scientific teams within an institution 17research on methodology and measurement of team science 13evaluation of team science and its impacts 58evaluating and learning from successful teams 50how to measure an increase in team science activity and collaboration at an institution, in comparison with other institutions 83how to evaluate existing and new tools 63how network information can provide insight into performance and evaluation of teams 88key performance indicators to encourage team science evaluation into individual development and professional growth 3comparing the effects of team science versus traditional science in advancing scientific knowledge 79infrastructures to capture relevant data to better assess team science outcomes 52to assess whether the findings produced by team science are more broadly disseminated, as compared to traditional science 18social network analysis of scientific teams 77strengthening the research methods for studying scientific teams (e.g., using quasi-experimental methods) 32how to use team science approaches and methods in the investigation of team science 29economic value created by team science 4how to demonstrate an effective team in a grant proposal 64the availability of organizational structure data as a data source 22approaches for capturing the expertise of team science leaders 2 3 4 8 13 17 18 22 29 32 46 50 52 58 63 64 65 69 77 79 83 88 89

26 Structure & Context for Teams 41whether collaborative spaces for team science encourage collaboration 54use and impact of community-based organizations and community clinical practices in teams 70what types of team organizations are best at facilitating team science 42the impact of team size on process and outcomes in team science 11the effects of the type and complexity of research question on team science 40how research networking tools can enhance team science 9the relationship between productivity and the composition of teams 67the effect of research centers in promoting a team science approach 20the relationships among creativity, innovation and the composition of teams 53contextual/situational factors that influence the effectiveness of team collaboration 30keys for success in team science 56effects of sustained, hard team work 7status of the team as it appears to external individuals and groups 28a study of team science outcomes with junior versus senior Pis 14how the changing ecology and structure of teams influence future scientific collaborations 37the network characteristics of productive science team members and subgroups 57how team dynamics can impact science 7 9 11 14 20 28 30 37 40 41 42 53 54 56 57 67 70

27 Characteristics & Dynamics of Teams 91status differences and power dynamics within the team 44team member interchangeability 21how roles in teams are defined and communicated, and by whom 26the influence of research team morale 60personal and behavioral factors in team science collaborations 94different types of conflicts that occur in scientific teams and how to address these effectively 23communication styles in teams 84leadership characteristics that drive effective team science 68optimal team composition (e.g., specialists, generalists, boundary spanners) to enable use of diverse expertise. 87social skills and competencies required for successful team science 27the psychological and personality factors associated with being an effective team scientist 47team member physical proximity (co-location) 24ideal composition of scientific teams 6heterogeneity of team membership 39how teams grow, shrink, expire over time 48issues to consider when initiating or building a new team 43collaborative readiness factors 49finding potential/likely research collaborators 34what factors contribute to the development of trust in different collaborations 95why people join teams 51gender differences in team contributions 6 21 23 24 26 27 34 39 43 44 47 48 49 51 60 68 84 87 91 94 95

28 Management & Organization for Teams 76value of rotating team leadership 93the management of scientific teams 78how to sustain scientific teams 61membership in multiple, potentially overlapping, potentially conflicting teams 35organizational policies that foster team science 66virtual organizations and team science 86formal vs. informal organizational structures of institutions 1types of organizational structures of team science 71disciplinary language barriers in team science 1 35 61 66 71 76 78 86 93

29 Institutional Support & Professional Development for Teams 31incentives and incentive systems for team science 74resources and infrastructure needed within and across institutions to promote collaboration and team science 25how the university tenure and promotion system can be restructured to encourage team science 80processes and methods that encourage and support teams (e.g., group activities, scientific conferences, grant opportunity distribution, systems-based approaches 85relationships between team science in the academy and industry 55timing, with regards to investigator career stage, in team science 92ethical issues in conducting team science (e.g. intellectual property ownership, defining collaborative relationships; attributing credit for work) 5training and education issues in team science 75funding to support the science of team science, research on team science 81co-authorship and multi-PI authorship in team science 12the effects of team science on the scientist's work and career 90use of collaborative computerized tools to support and enhance team science 16individual benefit/risk analysis to engaging in team science 5 12 16 25 31 74 75 80 81 85 90 92

30 Disciplinary Dynamics & Team Science 82variations in team science related to disciplinarity 38how to overcome disciplinary traditions to move toward interdisciplinary traditions 59using team science and interdisciplinary research to support emerging areas of science 36applying what is known about teams in different disciplines (e.g., management) and contexts (e.g., international) 33relationships and connections between multi-, inter- and transdisciplinary research efforts and team science 19how best to disseminate findings and best practices from the science of team science 10understanding differences between intra- vs. interinstitutional scientific teams 10 19 33 36 38 59 82

31 Definitions & Models of Team Science 72theories and models of team science 45best practices of team science 73developing testable hypotheses about team science 62the definitions of team, scientific team, and team science 15definition of different types of disciplinarity (interdisciplinary; multidisciplinarity; transdisciplinarity) 15 45 62 72 73

32 Cluster Map Management & Organization for Teams Characteristics & Dynamics of Teams Definitions & Models of Team Science Institutional Support & Professional Development for Teams Disciplinary Dynamics & Team Science Measurement & Evaluation of Team Science Structure & Context for Teams

33 Regional Interpretation Support The Team Nuts & Bolts Meta-Issues Management & Organization for Teams Characteristics & Dynamics of Teams Definitions & Models of Team Science Institutional Support & Professional Development for Teams Disciplinary Dynamics & Team Science Measurement & Evaluation of Team Science Structure & Context for Teams

34 Ratings and Pattern Matches

35 Importance Point Rating Map Management & Organization for Teams Characteristics & Dynamics of Teams Definitions & Models of Team Science Institutional Support & Professional Development for Teams Disciplinary Dynamics & Team Science Measurement & Evaluation of Team Science Structure & Context for Teams 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 Point Legend Layer Value 5 4.07 to 4.44 4 3.70 to 4.07 3 3.33 to 3.70 2 2.95 to 3.33 1 2.58 to 2.95

36 Importance Cluster Rating Map Management & Organization for Teams Characteristics & Dynamics of Teams Definitions & Models of Team Science Institutional Support & Professional Development for Teams Disciplinary Dynamics & Team Science Measurement & Evaluation of Team Science Structure & Context for Teams Cluster Legend Layer Value 1 3.43 to 3.49 2 3.49 to 3.55 3 3.55 to 3.61 4 3.61 to 3.68 5 3.68 to 3.74

37 Top Ten Statements By Average Importance 8measurement of key constructs (e.g., collaboration, disciplinarity, team effectiveness, personal/behavioral characteristics, team processes, readiness; synergy, productivity, shared knowledge) (4.44) 30keys for success in team science (4.23) 13evaluation of team science and its impacts (4.22) 45best practices of team science (4.16) 65measuring effectiveness of team science on multiple levels: individual team, impact of research, effectiveness of team science funding programs, etc. (4.16) 2how to evaluate success of team science-based research centers (4.14) 35organizational policies that foster team science (4.13) 9the relationship between productivity and the composition of teams (4.11) 3comparing the effects of team science versus traditional science in advancing scientific knowledge (4.08) 74resources and infrastructure needed within and across institutions to promote collaboration and team science (4.03)

38 Gender r =.5 FemaleMale 3.8 3.46 3.66 3.35 Institutional Support & Professional Development for Teams Characteristics & Dynamics of Teams Management & Organization for Teams Structure & Context for Teams Disciplinary Dynamics & Team Science Definitions & Models of Team Science Institutional Support & Professional Development for Teams Measurement & Evaluation of Team Science

39 Education r =.25 Bachelors or MastersDoctorate 3.7 3.18 3.75 3.47 Characteristics & Dynamics of TeamsDefinitions & Models of Team Science Management & Organization for TeamsCharacteristics & Dynamics of Teams Institutional Support & Professional Development for Teams Management & Organization for Teams Structure & Context for Teams Disciplinary Dynamics & Team Science Definitions & Models of Team ScienceInstitutional Support & Professional Development for Teams Measurement & Evaluation of Team Science

40 Professional Experience r =.7 Below MedianAbove Median 3.85 3.5 3.61 3.33 Characteristics & Dynamics of Teams Management & Organization for TeamsStructure & Context for Teams Disciplinary Dynamics & Team ScienceManagement & Organization for Teams Institutional Support & Professional Development for Teams Structure & Context for TeamsDefinitions & Models of Team Science Disciplinary Dynamics & Team Science Measurement & Evaluation of Team Science

41 Employment Sector r =.69 AcademicsGovernment 3.8 3.55 3.67 3.27 Characteristics & Dynamics of Teams Management & Organization for TeamsStructure & Context for Teams Institutional Support & Professional Development for Teams Management & Organization for Teams Disciplinary Dynamics & Team Science Structure & Context for TeamsInstitutional Support & Professional Development for Teams Definitions & Models of Team Science Measurement & Evaluation of Team Science

42 Team Science Focus r = -.08 PracticeScience 3.73 3.41 3.82 3.4 Characteristics & Dynamics of TeamsDefinitions & Models of Team Science Institutional Support & Professional Development for Teams Characteristics & Dynamics of Teams Management & Organization for TeamsDisciplinary Dynamics & Team Science Structure & Context for TeamsManagement & Organization for Teams Disciplinary Dynamics & Team ScienceStructure & Context for Teams Measurement & Evaluation of Team ScienceInstitutional Support & Professional Development for Teams Definitions & Models of Team ScienceMeasurement & Evaluation of Team Science

43 Experience with Teams r = -.8 None1 or 2 3.67 3.45 4 3.28 Disciplinary Dynamics & Team Science Institutional Support & Professional Development for Teams Structure & Context for Teams Measurement & Evaluation of Team Science Definitions & Models of Team Science Management & Organization for Teams Characteristics & Dynamics of Teams r =.43 3 - 5 3.67 3.41 r =.8 More Than 5 3.77 3.42 Characteristics & Dynamics of Teams Management & Organization for Teams Disciplinary Dynamics & Team Science Structure & Context for Teams Institutional Support & Professional Development for Teams Definitions & Models of Team Science Measurement & Evaluation of Team Science

44 Academic Discipline r =.48 SocialPhysical r =.4 Biomedical 3.96 3.56 3.77 3.44 Management & Organization for Teams Characteristics & Dynamics of Teams Structure & Context for Teams Disciplinary Dynamics & Team Science Definitions & Models of Team Science Institutional Support & Professional Development for Teams Measurement & Evaluation of Team Science r =.2 Humanities 3.67 3.23 r =.74 Engineering/Math 4.47 3.59 3.95 2.9 r = -.46 Business Administration 3.5 2.64 Definitions & Models of Team Science Disciplinary Dynamics & Team Science Characteristics & Dynamics of Teams Structure & Context for Teams Management & Organization for Teams Institutional Support & Professional Development for Teams Measurement & Evaluation of Team Science

45 Using the Maps: The Science of Team Science Conference Program

46 Regional Interpretation Support The Team Nuts & Bolts Meta-Issues Management & Organization for Teams Characteristics & Dynamics of Teams Definitions & Models of Team Science Institutional Support & Professional Development for Teams Disciplinary Dynamics & Team Science Measurement & Evaluation of Team Science Structure & Context for Teams

47 Support The Team Nuts & Bolts Meta-Issues Management & Organization for Teams Characteristics & Dynamics of Teams Definitions & Models of Team Science Institutional Support & Professional Development for Teams Disciplinary Dynamics & Team Science Measurement & Evaluation of Team Science Structure & Context for Teams Regional Interpretation - Panel X1

48 Support The Team Nuts & Bolts Meta-Issues Management & Organization for Teams Characteristics & Dynamics of Teams Definitions & Models of Team Science Institutional Support & Professional Development for Teams Disciplinary Dynamics & Team Science Measurement & Evaluation of Team Science Structure & Context for Teams Regional Interpretation – Panel X2

49 Support The Team Nuts & Bolts Meta-Issues Management & Organization for Teams Characteristics & Dynamics of Teams Definitions & Models of Team Science Institutional Support & Professional Development for Teams Disciplinary Dynamics & Team Science Measurement & Evaluation of Team Science Structure & Context for Teams Regional Interpretation – Panel X3

50 Support The Team Nuts & Bolts Meta-Issues Management & Organization for Teams Characteristics & Dynamics of Teams Definitions & Models of Team Science Institutional Support & Professional Development for Teams Disciplinary Dynamics & Team Science Measurement & Evaluation of Team Science Structure & Context for Teams Regional Interpretation - Panel X4

51 Support The Team Nuts & Bolts Meta-Issues Management & Organization for Teams Characteristics & Dynamics of Teams Definitions & Models of Team Science Institutional Support & Professional Development for Teams Disciplinary Dynamics & Team Science Measurement & Evaluation of Team Science Structure & Context for Teams Regional Interpretation - Panel X5

52 Support The Team Nuts & Bolts Meta-Issues Management & Organization for Teams Characteristics & Dynamics of Teams Definitions & Models of Team Science Institutional Support & Professional Development for Teams Disciplinary Dynamics & Team Science Measurement & Evaluation of Team Science Structure & Context for Teams Regional Interpretation - Panel X6

53 Support The Team Nuts & Bolts Meta-Issues Management & Organization for Teams Characteristics & Dynamics of Teams Definitions & Models of Team Science Institutional Support & Professional Development for Teams Disciplinary Dynamics & Team Science Measurement & Evaluation of Team Science Structure & Context for Teams Regional Interpretation - Panel X7

54 Thanks The Concept Mapping Steering Committee –Cath Kane –Holly Falk-Krzesinski –Steve Fiore –Dan Stokols –Kara Hall –Bonnie Spring –Noshir Contractor For more information –www.conceptsystems.comwww.conceptsystems.com –Bill Trochim: wmt1@cornell.eduwmt1@cornell.edu


Download ppt "Science of Team Science A Concept Mapping Project William M. Trochim Presentation to First Annual Conference on the Science of Team Science Northwestern."

Similar presentations


Ads by Google