1. 1 Best of 2008: Research findings you may have missed. Ryan Watkins, Ph.D. George Washington University www.gwu.edu/~etl www.ryanrwatkins.com

2. 2 Session Objectives Participants will be able to… Summarize leading research articles from multiple disciplines; Translate research finding into useful HPT applications; and Apply research findings in their daily practice of HPT.

3. 3 Best of 2008??? Step 1: Surveyed 20 researchers and faculty Both ISPI and non-ISPI members Varied disciplines HRD, OD, ISD, Management, Psychology No specific criteria for best Step 2: Reviewed diverse journals from 2008 Step 3: Based on initial input, sent preliminary list of articles back to researchers and faculty Step 4: Used their input to create a final list.

4. 4 Reminders Only a summary of each research article is being provided, please read the whole article if you are interested in the details. Presenting them in alphabetical order, not order of importance. To maintain copyright I can not provide copies of the full articles. The HPT Implications are my applications of the findings, you may have your own and the person sitting next to you may see different applications.

5. 5 Article One Title: An Examination of the Relationship Among Structure, Trust, and Conflict Management Styles in Virtual Teams Authors: Xiaojing Liu, Richard J. Magjuka, and Seung-hee Lee Publication: Performance Improvement Quarterly (an ISPI publication)

6. 6 Article One – Liu, Magjuka & Lee Purpose: Examine the roles of team structure, trust, and conflict variables in the effectiveness of virtual teamwork. Sample Hypotheses: – Nonhierarchical structures outperform hierarchical structures on their virtual team performance in complex task environments. – Trust will be positively associated with virtual team performance. – A collaboration conflict management style in virtual teams is positively associated with virtual team satisfaction.. – Hierarchical teams have a higher level of trust than nonhierarchical teams.

7. 7 Article One – Liu, Magjuka & Lee Variables: A 22-item questionnaire contained three dimensions: trust, conflict management resolution, and teamwork satisfaction. Methods: – Students enrolled in a capstone management course as part of an online MBA program at a large Midwestern university. – Subjects were separated into project groups of four to six people to work on an online simulation project. – Hierarchical (17 teams) and non-hierarchical (21 teams) team structures were assigned to the teams. – N = ~208 participants (some were dropped for teams not following the instructions, but the final N was not given).

8. 8 Article One – Liu, Magjuka & Lee Findings: – Team structure had a significant effect on team performance (with nonhierarchical leading to greater performance). – Team structure did not have a significant effect on team satisfaction. – Trust had a significant effect on team satisfaction, but not performance. – Collaborative conflict management had a significant effect on team satisfaction. HPT Implications: – In virtual teams, structure seems to have a greater influence on team performance, while social processes (i.e., team trust and conflict management) has greater influence on member satisfaction. – Even in a virtual team that is formed for a short time, it is worthwhile to facilitate swift relationship building through proactive communicative actions and to develop shared goals…

9. 9 Article Two Title: Feedback Acceptance in Developmental Assessment Centers: The role of feedback message, participant personality, and affective response to the feedback session Authors: Suzanne T. Bell and Winfred Arthur Jr. Publication: Journal of Organizational Behavior

10. 10 Article Two – Bell and Arthur Purpose: – Examine the role of the affective reaction to the feedback session in mediating the relationship between the assessor ratings and feedback acceptance. Hypotheses: – The relationship between assessor ratings and feedback acceptance will be mediated by the affective reaction to the feedback session. – Participant self-ratings will moderate the relationship between assessor ratings and feedback acceptance. – Three additional hypotheses related to the moderating role of personality characteristics (extroversion, emotional stability, and agreeableness).

11. 11 Article Two – Bell and Arthur Variables: – Assessor ratings, affective reaction to the feedback session, and feedback acceptance. – Participants self-ratings of their performance. – Participant personality characteristics of extraversion, emotional stability, and agreeableness as moderators. Methods: – Two organizations: First: Career development program intended to help mid-level managers develop their leadership skills. Second: Professional school of public administration which was training individuals for leadership and managerial positions in government or public service. – N = 141 participants, working with 35 trained assessors.

12. 12 Article Two – Bell and Arthur Findings: – Assessor ratings were positively related to feedback acceptance and this relationship was partially mediated by the affective reaction to the feedback session. Participants self-ratings, extroversion, and emotional stability did not serve as a moderator of this relationship. Participants agreeableness (good-natured, flexible, trusting, cooperative, and tolerant dispositions) did serve as a moderator. HPT Implications: – Performance feedback is essential to improving performance, and the context in which that feedback is given is very important to acceptance. – Build rapport before giving feedback (for example, light conversation before and after).

13. 13 Article Three Title: Implications of Domain-General Psychological Support Skills for Transfer of Skill and Acquisition of Expertise Authors: David W. Eccles and Paul J. Feltovich Publication: Performance Improvement Quarterly (an ISPI publication)

14. 14 Article Three -- Eccles & Feltovich Purpose: – Little research has been directed at identifying psychological skills that (1) support learning and performance and (2) are domain-general and thus can be applied in a variety of novel learning and performance settings.

15. 15 Article Three -- Eccles & Feltovich Variables: – Psychological Support Skills Domain-general skills that support meta-cognition. Mental imagery and rehearsal skills, self-talk management skills, goal-setting skills, relaxation skills, and planning, preparing, and organizing skills. – Meta-Cognition Ability to regulate ones psychological state in order to attain and subsequently maintain a state conducive to learning and performance. Sustain a high level of self-confidence, sustain a high level of motivation, control anxiety and cope with stress, control concentration and attention, and organize the learning environment and the learners time and energy. – Learning and Performance Methods: – Model development (early stages).

16. 16 Article Three -- Eccles & Feltovich HPT Implications: – Experts typically prepare over a 10-year period that includes upward of 10,000 hours of practice. – Transfer of domain-specific skills gets more challenging as performance gets more complex (i.e., tasks have fewer similarities). – Developing domain-general psychological support skills may be a good complement to meta-cognitive skills and weak (i.e., non-task specific) problem- solving methods for improving performance.

17. 17 Article Four Title: Improving the quality of evaluation participation: a meta-evaluation Authors: Darlene Russ-Eft and Hallie Preskill Publication: Human Resource Development International

18. 18 Article Four -- Russ-Eft & Preskill Purpose: – Describe the successes and failures in obtaining desired participation as of one organization attempted to implement several beta-test evaluations of a newly designed training program. Sample Research Questions: – Why did some clients not fulfill their commitment to complete all of the evaluations requirements? – What kind of incentives would have made clients more likely to participate fully in the evaluation? – For those clients who participated more fully in the evaluation process, what factors contributed to their participation?

19. 19 Article Four -- Russ-Eft & Preskill Variables: – A newly designed customer service program that replaced two older customer service programs the organization used to offer. – Pre-test, post-test, post-course reaction survey, 3 to 4 weeks post-course follow-up survey and telephone interviews. Methods: – Eight organizations (received free training in return), all of which has very low response rates to post-training surveys. – Meta-evaluation (i.e., an evaluation of an evaluation) using a case study method. – Ten 45-minute telephone interviews were conducted with 24 individuals.

20. 20 Article Four -- Russ-Eft & Preskill Findings: – Individual incentives probably would not have helped, free training was enough. – Sample reasons for none response: They were too busy. They left the organization. They changed jobs within the organization. The office was shut down. A lack of manager follow.

21. 21 Article Four -- Russ-Eft & Preskill HPT Implications: – Respondents must know that they have the authority to respond for the organization. Key sponsors within an organization can help communicate this authority. – Respondents and their organization must have the capacity to participate fully in the beta-test (including having time available to participate in all aspects of the evaluation). – The motivation to participate in the evaluation must be built, including what is in it for them and the importance of their input.

22. 22 Reflection Activity Identify one take away from the five research articles previously presented Discuss your take away with the person sitting next to you

23. 23 Article Five Title: Institutional Economics and Human Resource Development Authors: Greg G. Wang and Rita L. Dobbs Publication: Advances in Developing Human Resources

24. 24 Article Five – Wang & Dobbs Purpose: – Expand the knowledge of institutional economics and its foundational role and direct applicability to HRD research and practice. – Institutional economics… is the study of relationships between the role of human institutions (i.e., organizations) and the related economic behaviors. considers the complex interactions of various institutions. For example, individuals, organizations, governments, socio- culture and history, rather than a simple demand–supply equation.

25. 25 Article Five – Wang & Dobbs Focus: – Orthodoxy neoclassical economics focuses on the allocation of scarce resources among alternative ends. It emphasizes equilibria, where the solutions to individual and firm maximization problems are reached. – Alternatively key assumptions of institutional economics include: 1. Institutions involve the interaction of agents with crucial information feedbacks. 2. Institutions have common conceptions and routines. 3. Institutions sustain and are sustained by shared conceptions and expectations. 4. Institutions have relatively durable, self-reinforcing, and persistent qualities. 5. Institutions incorporate values and processes of normative evaluation.

26. 26 Article Five – Wang & Dobbs Assumptions: Assumption of InstitutionalAssumptions of Neoclassical Labor market Imperfect labor market with underemployment as an outcome Labor market is competitive and reaches full employment in the long-run Jobs Institutional rules and customs allocate employees jobs within unequal job and training structures Job allocation is by market clearing prices or wages HRD HRD increases the value of employee to the organization, and the interventions is embedded in the work process as well as taught by coworkers Training is differentiated by types of skills, i.e., specific vs. general, which determines who pays for the investment and who receives the return

27. 27 Article Five – Wang & Dobbs Assumptions: Assumption of InstitutionalAssumptions of Neoclassical Labor Market Focus Internal labor market – promotion, OJT, career ladders, internal hirers, training, and other factors are accounted for decisions. External Labor Market - pricing, allocating, hiring, training, and other decisions are controlled directly by economic variables. Learning Learning is a developmental and reconstitutive process. Learning typically takes place through and within social structures and involves adaptation to changing circumstances. Such adaptations mean the reconstitution of the individuals involved. Learning is the cumulative discovery of preexisting blueprint information, stimulus and response, or the updating of subjective probability estimates in light of incoming data.

28. 28 Article Five – Wang & Dobbs HPT Implications: – Both Institutional and Neoclassical economic theories are at play in most organizations, and their application plays an important role in the application of HPT and HRD in organizations. – We should be familiar with both, and understand the consequences of each as they applied in organizations (especially in these economic conditions). – Realize that we may be coming an Institutional Economics perspective but others in our organizations may have a Neoclassical perspective.

29. 29 Article Six Title: Investigating the Success of Decision Making Processes Authors: Paul C. Nutt Publication: Journal of Management Studies

30. 30 Article Six -- Nutt Purpose: – Effectiveness of two decision making processes… – Idea-imposition which caters to the interests of powerful stakeholders by looking for ready-made solutions they can support. A ready-made solution allows the DM to visualize a course of action and its ramifications before commitments are made. – Discovery (i.e., HPT) which gathers intelligence about needs, specify desired results, uncover options, evaluate options according to their benefits, and implement by working with interest groups and their perceptions to install the most beneficial option.

31. 31 Article Six -- Nutt Sample Hypotheses: – The prospect of success increases when decision makers follow a discovery process compared to an idea-imposition process. – The prospect of success declines when decision makers use an emergent opportunity process, compared to a discovery process. – For each of the contingencies considered, such as high and low urgency, the prospect of success will be greater when decision makers follow a discovery process, compared to the three other processes.

32. 32 Article Six -- Nutt Variables: – Discovery and the emergent opportunity variation. – Idea-imposition and the redevelopment variation. – Frequency of use in relation to success. – Controlling for context and decision type.

33. 33 Article Six -- Nutt Methods: – Analyzed 202 decisions about: financing, internal operations/controls, personnel policy, marketing, buildings, technologies, and reorganizations. – Profit and non-profit; top and mid-level managers – Interviews with decision makers used to determine actions, and decision making process. – Indicators of effectiveness and efficiency are used to measure success, with at least 2 raters on each decision.

34. 34 Article Six -- Nutt Findings: – Discovery had far more success than idea imposition. – Decision makers are drawn to idea-imposition because it seems timely and pragmatic, the results however indicate little timeliness, low quality, and very low adoption rates compared to discovery. – Discovery produced far better outcomes, with 90% sustained adoptions, 85% complete adoptions, rated as good to excellent, completed in nearly half the time. – Emergent opportunity processes fell a little short of discovery in effectiveness, and were inefficient.

35. 35 Article Six -- Nutt HPT Implications: – Decision made using a systematic HPT process that begins with a needs assessment (without a solution) are more likely to produce desired results, results are more likely to be maintained, and results will be achieved more efficiently. – It is tempting, but do not abandon your systematic processes once you think a good solution is identified (i.e., emergent opportunity), this is less effective and significantly delays results.

36. 36 Article Seven Title: Long work hours: a social identity perspective on meta-analysis data Authors: Thomas Ng and Daniel Feldman Publication: Journal of Organizational Behavior

37. 37 Article Seven -- Ng & Feldman Purpose: – Understand the factors that contribute to longer work weeks and the relationships among hours worked, job performance, and employee well-being. Sample Hypotheses: – Job security, organizational support, job autonomy, opportunities for learning, organizational tenure, job level, current salary, number of promotions, career satisfaction, educational level, general work experience, international work experience, and social networking are each positively related to employees hours worked. – Hours worked are positively related to job stress, mental strain, physical health problems, work injuries, and use of tobacco, alcohol, and drugs.

38. 38 Article Seven -- Ng & Feldman Methods: – Meta-analysis. – 20 years of journal articles and 7 dissertations. – Organizational science, applied psychology, and sociology journals. – 199 relevant articles, containing a total of 222 independent samples.

39. 39 Article Seven -- Ng & Feldman Findings: – Average number of hours worked was 42.4 (SD is 10.2 hours). – Work hours were positively related to job stress and mental strain. – Once a very high state of stress is reached, each new hour work creates marginally less additional stress. – One additional work hour in an already jammed week may exponentially increase work–family conflict. – The relationship of career satisfaction and educational level with long work hours were stronger for women than for men. – With some exceptions, the results here provide some support for the position that working long hours is negatively associated with work effectiveness and work attitudes.

40. 40 Article Seven -- Ng & Feldman HPT Implications: – Long work hours can increase stress, mental strain, and family-to-work conflict. – The factors leading to long work hours are numerous, thus if we want to manage work hours (increasing or decreasing) you will have to account for multiple variables.

41. 41 Article Eight Title: Organizational diversity, integration and performance Authors: Andrew H. Van De Ven, Russel W. Rogers, John P. Bechara, And Kangyong Sun Publication: Journal of Organizational Behavior

42. 42 Article Eight -- Van De Ven, et. al Purpose: – Examine how encouraging diverse perspectives (for example, around a change) influences the relationship of varied models of organizing on performance. Hypothesis: – Integrative behavior moderates the relationship between organizational diversity in perspectives and performance.

43. 43 Article Eight -- Van De Ven, et. al Variables: – Integrative behavior as the degree to which employees perceive that their organization encourages different perspectives and synthesizes those perspectives in ongoing day-to-day activities (e.g., openness to ideas and change involvement). – Models of organizing: System/Bureaucracy Model; Market Model; Profession Model; and Community Model. – Performance: Clinic productivity, net income, and patient satisfaction. Methods: – 37 primary healthcare clinics go through restructuring changes. – Surveys and secondary-data analysis.

44. 44 Article Eight -- Van De Ven, et. al Findings: – Although the evidence is not as strong, results show that the main effect of diverse organizing models on performance is positive. – Diverse mental models for organizing among employees was positively related to performance, and this relationship was substantially stronger when integration processes for managing diversity are taken into account. HPT Implications: – Processes that involve employees in decision making, encourage open debate and integrative methods of conflict resolution, and diversity of perspectives are associated with higher performance. – Do not focus solely on developing a single vision of change, but rather engage people with alternative perspectives. – Supports Framing Organizations by Bolman and Soft Systems Theory by Checkland.

45. 45 Reflection Activity Identify one take away from the five research articles previously presented Discuss your take away with the person sitting next to you

46. 46 Article Nine Title: Performance Implications Of Delayed Competitive Responses: Evidence From The U.S. Retail Industry Authors: Jens L. Boyd And Rudi K. F. Bresser Publication: Strategic Management Journal

47. 47 Article Nine – Boyd and Bresser Purpose: – Contribute to ongoing efforts to develop theory on first mover and follower advantages by revisiting the consequences of fast versus delayed competitive responses. Research Question: – What is the nature of the response timing- performance relationship?

48. 48 Article Nine – Boyd and Bresser Hypotheses: – Curvilinear relationship between response delay and responder performance (i.e., on average, there will be lower success for fast responders, higher success for responders with intermediate delays, and lower success for late responders). – Linear relationship between response delay and first mover performance (i.e., as response delays increase, the impact on the performance of first mover decreases).

49. 49 Article Nine – Boyd and Bresser Variables: – Response delay (number of working days between action and reaction). – Stock price effects (first mover and responder). – Control variables of imitation, tactical actions, firm size, financial performance, age, industry concentration, and industry sales growth. Methods: – The 17 largest U.S. department stores from 1994 to 2000. – Structured content analysis of 982 press articles. – 370 competitive moves including 105 responses.

50. 50 Article Nine – Boyd and Bresser Findings: – Most responses were imitations, indicating that many responders intend to copy to catch up, potentially further escalating competition. – As responders move too quickly or too slowly, this impacts their performance. – The performance effects of response delays for responders and first movers do not mirror each other. HPT Implications: – Be aware of the risk of responding too fast to competition. – Response timing requires a balance between the risks of premature entry against missed opportunity.

51. 51 Article Ten Title: Task knowledge overlap and knowledge variety: the role of advice network structures and impact on group effectiveness Authors: Sze-Sze Wong Publication: Journal of Organizational Behavior

52. 52 Article Ten -- Wong Purpose: – Study 1 examines whether advice networks and group knowledge affect subsequent group performance. – Study 2 was conducted on 40 knowledge-intensive work groups from three organizations.

53. 53 Article Ten -- Wong Sample Hypotheses: – The higher the knowledge overlap and variety in a group, the higher the group effectiveness. – The higher the density of the internal advice network (i.e., the more each group member seeks advice from other members), the higher the degree of knowledge overlap in a group. – The higher the centralization of the internal advice network (i.e., a few key influential group members), the lower the degree of knowledge variety in a group.

54. 54 Article Ten -- Wong Variables: – Knowledge variety is the range of different types of task knowledge in a group. – Knowledge overlap is the extent to which group members have common task knowledge – Internal and external social advice network structures (rather than social or friendship networks). Methods: – Study 1: Sample of 80 undergraduate students groups involved in a complex business simulation task. – Study 2: 50 work groups from a hospital, an industrial diversified firm, and a high-technology firm.

55. 55 Article Ten -- Wong Findings: – Knowledge overlap was not significantly associated with group effectiveness in both studies. – Knowledge variety was positively associated with group effectiveness in both studies. – Internal network centralization was negatively associated with knowledge variety in both studies. – Internal network centralization was positively related to group effectiveness in Study 1, but not Study 2.

56. 56 Article Ten -- Wong HPT Implications: – Promote knowledge breadth for groups involved in knowledge-intensive work (such a cross-disciplinary teams). – Foster ties with many different individuals, both within and outside of your group, to promote knowledge variety. – Network with external groups.

57. 57 Article Eleven (BONUS) Title: The Effect of Performance Support and Training as Performance Interventions Authors: Frank Nguyen & James Klein Publication: Performance Improvement Quarterly (an ISPI publication)

58. 58 Article Eleven -- Nguyen & Klein Research Questions: – What combination of performance support and training maximizes user performance? – What combination of performance support and training do users prefer? – Do users access performance support more if they have not had training prior to task performance? – What combination of performance support and training minimizes the time to complete a task?

59. 59 Article Eleven -- Nguyen & Klein Variables: – EPSS, EPSS + web-based training, and web-based training (with no partial-task practice in web-based training). – User performance on the task; user attitudes; use of EPSS; time in training; and time to complete task. Methods: – Focus on use of tax preparation software. – Posttest-only control group design, using assessments and interviews. – 78 participants from various organizations.

60. 60 Article Eleven -- Nguyen & Klein Findings: – Training & EPSS and EPSS-only had significantly higher scores on the task than training-only. – Time on task EPSS-only = 26 minutes Training & EPSS = 31 minutes Training only = 1.5 hour HPT Implications: – Adding performance support to training can significantly increase performance. – EPSS may be sufficient by itself in many situations, though the addition of training demonstrated some attitudinal benefits. – Finding the right mix is key.

61. 61 Reflection Activity Identify one take away from the four research articles previously presented. Discuss your take away with the person sitting next to you.

62. 62 Survey Was Sent To… Richard Clark (USC) Richard Swanson (Univ. Texas, Tyler) Darlene Russ-Eft (Oregon State) Tim Hatcher (NCSU) Rob Branch (UGA) Jim Klein (ASU) Steven Condly (UCF) Jan Visser Jeroen Van Merrienboer Gordan Rowland (Ithica ) Jim Altshuld (OSU) Harold Stolovich Ingrid Guerra-Lopez (Wayne) John Wedman (Missouri) Scott Schaffer (Purdue) Maria Cseh (GWU) Julia Storberg-Walker (NCSU) Hillary Leigh Tony Marker (BSU) Yonnie Chyung (BSU)

63. 63 Best of 2008: Research findings you may have missed. Ryan Watkins, Ph.D. George Washington University www.gwu.edu/~etl www.ryanrwatkins.com