1. Leadership of your PhD Anders Dysvik, Ph.D. Professor of Organizational Behavior BI Norwegian Business School

2. Spend 3 minutes discussing facets of research quality How rigorous is the study and the report? How original and new is the contribution? How relevant is it within the field? How usefulness is it in society? Can we pick and choose among these requirements - or are they interdependent?

3. Requirements I Prepared for replication Empirical support or lack of rejection Precision regarding constructs, level of analysis, form of relationships, and generalizability A unifying conceptual framework Falsifiable statements and freedom from prejudice

4. ”…when properly performed, the rain dance will make rain fall from the skies”.

5. ”…when properly implemented, incentive motivators are effective mechanisms for enhancing individual performance”.

6. Peterson, S. J., & Luthans, F. (2006). The impact of financial and nonfinancial incentives on business- unit outcomes over time. Journal of Applied Psychology, 91(1), 156-165.

7. Requirements II Generalizability theoretical statements are by definition about something more than single cases the less situation specific, object specific and time specific, the better Generalizability in terms of the number of different persons and objects how well a phenomenon is explained how many phenomena that are explained

8. The research problem/question More specific than the research topic Not given, they are discovered, found or constructed Something we don’t already know and that is possible to explore, investigate or answer with the means of scientific methodology Should solve a problem or gain relevant insight that develops theory or practice Explicitly related to relevant research literature(s) Evolving during the early phases Expressed in terms of relationships and/or questions

9. Positioning the research problem Novelty something we don’t already know typical, a lack of knowledge, i.e. “few studies have….” Contribution to theory and/or practice the reason why we should know something we don’t already know – lack of knowledge is not sufficient! “Accordingly, the purpose of this study was to test the relationship between …….. with the objective of contributing to …. by way of ….”

10. Research problem check list Is it clearly formulated? (formulation vs. content) Is it specific enough to be dealt with within existing time and resource constraints? Is it positioned – is it explained why it is interesting (cf. research quality)? Is it related to existing literature(s)? Are the hypotheses related to it?

11. What does it take to make a scientific contribution? Contribution in terms of theory empirical or both That something has not been investigated (well enough) yet is a necessary, but not a sufficient condition as there may be a number of good reasons for lack of research on a particular phenomenon

12. Some pitfalls you should avoid Not explicitly stating the contribution you set out to make Stating it, but not explicitly enough exactly how will the study increase our understanding of the phenomenon Stating it, but too late in paper Stating it in the beginning, but not in the discussion section Ending up with a “collection of variables” with several (too) minor contributions minor contributions can mentioned in the discussion section

13. Some pitfalls you should avoid Several (too) small contributions ending up with a collection of variables adding every variable that may explain variance pick one or two main contributions, leave minor ones to the discussion section Making a (too) small empirical contribution replications in contexts where there are no compelling reasons that you will not end up with the same findings/observations as in the original study need to explicitly state why and how the findings/observations should be different

14. The importance of active reading and writing Don’t buy everything you read tactics in the business of publish or perish Don’t use rules of thumbs as rules without understanding them statistical significance and α = 0.7 or 0.69 choices of statistical methods and analyses (e.g. second order models, EFA/CFA, lack of discriminant validity) Don’t write something you don’t understand as lack of clarity won’t disappear by itself Be an active reader and writer

15. Normative research problems How to conduct performance appraisal that increases employee performance? not suited for empirical testing an interesting study contains much more than a discussion of factors (different Xs) that may influence Y e.g. in order for performance appraisal to increase employee performance it is important to Reframe the research problem to contain something we don’t already know (enough about) e.g. whether specific factors of performance appraisal are related to employee performance

16. Research problems without variance in X when we ask whether X can explain Y Does an organization’s CSR practice influence employees’ affective commitment? impossible to investigate with only one organization because X is a constant and not a variable Look for differences between groups or units or significantly increase the number of organizations Or, investigate employee perceptions of X, knowledge about X, use of X etc. Does employee perceptions of an organization’s CSR practice influence their affective commitment?

17. Research problems where X does not exist yet Will implementing, increasing or changing X increase Y? Will increasing pay differences among employees lead to perceptions of injustice? It is better to investigate perceptions of the existing X (perceptions of or tolerance for pay differences) Or, investigate antecedents to X (then X becomes Y), which may be important to know before an organization implements, increases or otherwise changes X (e.g. differences between groups of employees)

18. Purely descriptive research problems Do managers and employees vary in their perceptions of an organization’s HR practices? not very interesting Extend the research problem and include factors that may explain a potential variance in perceptions of X

19. Quality of research publications The Harzing-list provides several rankings http://www.harzing.com/ Australian Business Deans’ Council Journal List http://www.abdc.edu.au/ Four levels: A*, A, B, C 11 Disciplines: Business and Management

20. A*-Journals (28/424 = 6.60%) Best or leading journal in its field Publishes outstanding, original and rigorous research that will shape the field Acceptance rates are typically low and the editorial board is dominated by leading scholars in the field or subfield, including from top institutions in the world Where relevant to the field or subfield, the journal has the highest impact factors or other indices of high reputation Acceptance rates usually less than 10%

21. A selection of A*-Journals Academy of Management Journal Academy of Management Review Administrative Science Quarterly British Journal of Industrial Relations California Management Review Decision Sciences Entrepreneurship: Theory and Practice Harvard Business Review Human Resource Management (US) Industrial Relations Journal of Applied Psychology Journal of Business Venturing Journal of Int. Business Studies Journal of Management Journal of Management Studies Journal of Operations Management Journal of Organizational Behavior Journal of Personality and Social Psych Product Innovation Management Journal of Vocational Behavior Leadership Quarterly Management Science Operations Research Organization Science Organization Studies Org. Behavior & Human Decision Proc. Personnel Psychology Strategic Management Journal

22. A-Journals (68/424 = 16.04%) Highly regarded journal in the field or subfield Publishes excellent research in terms of originality, significance and rigour, has competitive submission and acceptance rates, excellent refereeing process and where relevant to the field or subfield, has higher than average impact factors Not all highly regarded journals have high impact factors, especially those in niche areas Acceptance rates 10% - 20%

23. A selection of A-journals Academy of Management Perspectives Academy of Management Learning and Education Annals of Operations Research Applied Psychology: An Int. Review British Journal of Social Psychology Business Ethics Quarterly European Journal of Industrial Relations Family Business Review Group & Organization Management Human Performance Human Relations Human Resource Mgmt Journal (UK) Journal of Industrial Relations Journal of Occupational Health Psychology Journal of Occupational and Org. Psych. Journal of Small Business Management Journal of World Business Long Range Planning Management International Review Management Learning Organization Organizational Dynamics Organizational Research Methods Supply Chain Management Work and Occupations

24. B-Journals (97/424 = 22.88%) Well regarded journal in the field or subfield Publishes research of a good standard in terms of originality, significance and rigor and papers are fully refereed according to good standards and practices but acceptance rates are higher than for Tiers A* and A Depending on the field or sub-field, will have a modest impact factor and will be ISI listed Acceptance rates 20%-50%

25. A selection of B-journals Canadian Journal of Adm. Sciences Employee Relations European Business Review European Journal of Work and Organizational Psychology Human Resource Development Int. Human Resource Development Quart. Human Resource Development Review Human Resource Management Review Industrial Relations Journal Integrated Manufacturing Systems International Journal of Conflict Mngt. International Journal of Cross-Cultural Management Journal of Applied Behavioral Science Journal of Business and Psychology Journal of General Management Journal of Labour Research Journal of Management Education Journal of Management Inquiry Journal of Organizational Behavior Mngt. Journal of Org. Change Management Labor Studies Journal Labour and Industry Leadership and Org. Dev. Journal Management Communication Quarterly Nonprofit Management & Leadership Organization Development Journal

26. C-Journals (231/424 = 54.48%) A recognized journal Publishes research that is of a modest standard and/or is yet to establish its reputation because of its newness This tier is more inclusive than the others but only includes refereed journals Acceptance rates typically > 50% Remember that most journals are not listed!

27. A selection of C-journals Advances in Developing Human Res. Advancing Women in Leadership Asia-Pacific Journal of Human Res. Career Development International Culture and Organization Development and Learning in Organizations: An International Journal Employment Relations Record Equal Opportunities International European Business Journal European Management Journal European Management Review Human Systems Management Intern. Employment Relations Rev. International Journal of Management Intern. J. of Mngt. and Decision Making Intern. Journal of Management Reviews Intern. Journal of Organization Behaviour Intern. J. of Training and Development Intern. J. of Work Org. and Emotion Intern. Rev. of Industrial & Organisational Intern. Review of Women and Leadership Journal of Change Management Scandinavian Journal of Management

28. Relevance versus Rigor

29. Top journals = top research? Dissensus among reviewers interrater reliability typically less than 0.30 Low acceptance rates “Highly prestigious journals publish quite a few low-value articles, low prestige journals publish some excellent articles, and excellent manuscripts may receive successive rejections from several journals” Variation in article impact 12% of articles published in top journals are not cited less than 4% generate more than 100 citations reviewers are not able to predict the future impact of an article (r =.14) Glick, W. H., Miller, C. C., & Cardinal, L. B. (2007). Making a life in the field of organization science. Journal of Organizational Behavior, 28(7), 817-835.

30. Conclusions Don’t equate top journals with top research although usually more rigor (multiple data sources, better measures, longitudinal designs etc.) Don’t buy everything you read good journals also publish low quality research top scholars sometimes publish low quality research