Research you can use Judith Olson University of California Irvine

 Three recent events that inspired this “call to arms”  Your role in this discussion  What it means to have impact  Kinds of impact  Recap of what it means to have impact  Scope, Cost, Timeline…  Your pledge about making an impact

 Theory based on  The literature on teams  Own own observations and interviews of over 50 ▪ Science Collaborations ▪ Corporate virtual teams  To verify theory  Need data  Online survey with advice to motivate participation  They get the help and we get the data

 Web accessible assessment tool  Assesses  Strengths  Challenges  How to overcome the challenges

 NSF  Had us give a talk to Federal funders in general  “I have needed this for the last 10 years! Thank you.”  Teams who were assessed welcomed advice  “It drew out patterns in the way our members work that we were not conscious of, confirmed some of our impressions, and allowed us to hear frankly from our members.  …useful as an independent evaluation tool not tied to a funding agency or other review panel”.

 Object of study  …”to speed the translation of laboratory discoveries into treatment for patients.”  “from bench to bedside.”  National Institute of Health:  60 CTSA awards in 30 states plus DC

National Institutes of Health Since 2006 $733 M

“While most researchers know what is meant by Intellectual Merit, experience shows that many researchers have a less than clear understanding of the meaning of Broader Impacts.”

 Many of us came to this field to change the digital world  Technology had gone awry  Many early people attracted to HCI were “Children of the 60s”

 Our careers were caught up in the reward structures  Industry ▪ Create new products ▪ Disincentive to make findings available to others  Academia ▪ Publish new findings ▪ Stay on topic, build a reputation

 Where have all the impacts gone  Long time passing

 I will describe what I think it means to have impact  I will list a number of ways we do and can have an impact  You pledge…  The card on your seat ▪ What other ways can you have impact ▪ How are you going to have an impact ▪ Collected by SVs at the door as you leave

 What “counts”  Theory gets used  Downloads/views  Profits  Degrees/Education  Technologies  Lives changed  …..  Who is impacted?  Students  Developers  Consultants  Specific populations  The general public

 Scopes differ  You affect some people directly ▪ Interventions, teaching  You enable others to be better at making better products ▪ Toolkits  You set policy ▪ Affect a large number of people

 Time scales differ  Now ▪ e.g. Action research  1-3 years ▪ e.g., Publications  years ▪ e.g., Theory  Assessment Tools  years ▪ e.g., Cyberinfrastructure development  ? ▪ e.g., Policy (like SOPA/PIPA)

 Access?  Free ▪ Toolkits… ▪ Wizard ▪ ….  Fees ▪ Commercial Assessment Tools ▪ Products ▪ Educational degree ▪ ….

 Theories  Assessment tools  Technological innovations  Guidelines, templates, patterns, toolkits and standards  Policies  New media dissemination  Action research  Teaching and teaching materials  … What else? Which kinds of impacts will YOU make?

 “There is nothing so practical as a good theory” Kurt Lewin  “He who loves practice without theory  is like the sailor who boards ship without a rudder and compass and  never knows where he may cast” Leonardo Da Vinci

 Who  Other researchers  Consultants  Tool developers  How  Read and build on/test theory  Scope  Small at first  Time scale  1-3 or more years  Access  Free

 Theories  Assessment tools  Technological innovations  Guidelines, templates, patterns, toolkits and standards  Policies  New media dissemination  Action research  Teaching and teaching materials What else? Which kinds of impacts will YOU make?

 Collaboration Success Wizard  Globesmart  Myers-Briggs Personality Assessment  CogTool  …

 Based on academic theories of cultural differences ▪ David Matsumoto ▪ Handbook of Culture and Psychology  Like the Wizard, they collect data to adjust their assessments ▪ Recent upgrade used data from 400,000 users from over 60 countries

 Based on the work of Carl Jung  Developed further by Myers and Briggs  Like the GlobeSmart you can see differences in values and habits with people you interact with  Dimensions of discussion  Some professional help

 Based on work of Bonnie John ▪ Based on Card, Moran, & Newell ▪ GOMS and the Model Human Processor  A general purpose UI prototyping tool  It automatically evaluates your design with a predictive human performance model ▪ A “cognitive crash dummy”

▪ “You can compare expert use task time without recruiting participants…An excellent choice for completely new systems that don’t already have experts.”

 Who  General public  How  Take the assessment  Scope  Could be huge  Time scale  Immediate  Access  Some are free; some cost money

 Theories  Assessment tools  Technological innovations  Guidelines, templates, patterns, toolkits and standards  Policies  New media dissemination  Action research  Teaching and teaching materials What else? Which kinds of impacts will YOU make?

  3-D programming environment  For telling a story  Playing an interactive game  Teaching tool for introductory programming  Formally shown to improve learning and performance Randy Pausch

Caitlin Kelleher, 2006

 Using storytelling to make computer programming attractive to middle school girls  Storytelling Alice users  spent 42% more time programming  were more than three times as likely to sneak in extra time to continue working on their programs Caitlin Kelleher

 10% of the nation’s colleges now use Alice  An accompanying textbook, lessons, test banks  88% of “at risk” students who had Alice in a pre-CS1 course were retained through CS2  3.03 GPA

 iMuse  A requirements engineering environment where both developers and stakeholders could understand the flow Kristina Winbladh

 Hypertext Transfer Protocol HTTP  HTTP/1.1 spec ▪ Fielding, Gettys, Mogul, Frystyk and Berners-Lee  WebDAV extension  “Architecture of the Web” ▪ Fielding and Taylor

 Aspect Oriented Programming ▪ Difference lies in the power, safety and usability of the constructs provided  Original article downloaded 6,681 times  16,600 articles in Google Scholar with “Aspect Oriented Programming” Crista Lopes

 Who  Students  The general public  Other developers  How  Use the technology that makes things possible  Scope  Huge  Time scale  5-10 years  Access  Often free (though products cost money)

 Theories  Assessment tools  Technological innovations  Guidelines, templates, patterns, toolkits and standards  Policies  New media dissemination  Action research  Teaching and teaching materials What else? Which kinds of impacts will YOU make?

 All provide conventions  So there is little new to learn  Where things go, what they look like  Sometimes task flow guide

 What are they based on? Are they consistent? (Human Interface Guidelines)

 Principles, patterns and practices for improving use experience  Early instance: Christian Crumlish & Erin Malone Christopher Alexander

 Their effectiveness depends on ▪ The research they are based on ▪ The context in which they arose ▪ Their fit to the context they are being applied to

 UI Development environments  With extra features ▪ Highly interactive ▪ Graphical ▪ Direct manipulation ▪ Automatic undo ▪ Support for animation ▪ Gesture recognition  Amulet - C++  Garnet – Common Lisp, X11, and Mac Brad Myers

Keeping these up to date….

 Who  Developers  End users  How  Find and use relevant templates….  Scope  Speeds development, makes software consistent  Time scale  Immediate  Access  Free

 Theories  Assessment tools  Technological innovations  Guidelines, templates, patterns, toolkits and standards  Policies  New media dissemination  Action research  Teaching and teaching materials What else? Which kinds of impacts will YOU make?

 Stop Online Piracy Act (SOPA)/Protect Intellectual Property Act (PIPA)  Network neutrality  Participatory design in Scandinavia  Open access vs. commercial production of educational materials  Data sharing policies  ……

 Who  Everyone  How  Dictates what’s possible  Scope  Huge  Time scale  ?  Access  Who gets to be in the conversation?

 Theories  Assessment tools  Technological innovations  Guidelines, templates, patterns, toolkits and standards  Policies  New media dissemination  Action research  Teaching and teaching materials What else? Which kinds of impacts will YOU make?

 Total views = 14,660,471

 All videos viewed 6,813,795 times Hans Rosling A Swedish medical doctor, academic, statistician and public speaker. He is Professor of International Health at Karolinska Institute [2] and co-founder and chairman of the Gapminder Foundation, which developed the Trendalyzer software system.Karolinska Institute [2]Gapminder FoundationTrendalyzer

 Who  The public  Students  How  YouTube, TedTalks….  Scope  Huge  Time scale  Immediate  Access  Free

 Theories  Assessment tools  Technological innovations  Guidelines, templates, patterns, toolkits and standards  Policies  New media dissemination  Action research  Teaching and teaching materials What else? Which kinds of impacts will YOU make?

 Helping teachers of autistic children assess behavioral incidents  Helping caretakers and clinicians of preterm infants monitor their movement and other key factors Gilllian Hayes

 Who  Target population starting with a small group  How  New technologies to help critical situations  Scope  Small at first, larger as results are generalized  Time scale  Immediate  Access  Free

 Theories  Assessment tools  Technological innovations  Guidelines, templates, patterns, toolkits and standards  Policies  New media dissemination  Action research  Teaching and teaching materials What else? Which kinds of impacts will YOU make?

 Undergraduate teaching  6,970 students in a career  Ph.D. students ~40 ▪ A multiplier because they go on to teach  Teaching materials  Books for classes  Cases, exercises

 Online resources that educate UsabilityFirst.com Hcibib.org Useit.com

 Teaching or action kits National Center for Women in Information Technology NCWIT

 NCWIT

David Evans & Sebastian Thrun

 Who  Students  How  Exposed to lectures, exercises, assessments  Scope  Digital media is the multiplier  Time scale  1-2 years  Access  Sometimes free; sometimes requires tuition

 Theories  Assessment tools  Technological innovations  Guidelines, templates, patterns, toolkits and standards  Policies  New media dissemination  Action research  Teaching and teaching materials  … What else? Which kinds of impacts will YOU make?

 Who is impacted  How  Scope  Time Scale  Access  Decisions you have to make…

 What “counts”  Theory gets used  Downloads/views  Profits  Degrees/Education  Technologies  Lives changed  …..  Who is impacted?  Students  Developers  Consultants  Specific populations  The general public

 Scopes differ  You affect some people directly ▪ Interventions, teaching  You enable others to be better at making better products ▪ Toolkits  You set policy ▪ Affect a large number of people

 Time scales differ  Now ▪ e.g. Action research  1-3 years ▪ e.g., Publications  years ▪ e.g., Theory  Assessment Tools  years ▪ e.g., Cyberinfrastructure development  ? ▪ e.g., Policy (like SOPA/PIPA)

 Is it free?  Yes ▪ Khan Academy ▪ Open Knowledge ▪ Standards, toolkits, patterns ▪ Wizard  No ▪ Udacity ▪ Meyers Briggs ▪ GlobeSmart ▪ Textbooks ▪ Degree programs ▪ Products

 How to translate our research to have broader impacts?  How to guarantee quality?  E.g. evidence based medicine  How to make it accessible?  How to evaluate impact?

 How careers are advanced now  Product innovation  Publications  Future  + Impact  It takes the evaluators to change the system  Promotion policy

 Theories  Assessment Tools  Popular technologies that become standards  Guidelines, templates, patterns, toolkits and standards  Policies  New media dissemination  Action Research  Teaching and teaching materials  … Student Volunteers will collect on the way out

 In the interest of potential impact  A video of this will appear on the ACM website and the ACM-W website 