The Application of Cognitive Processes to Organizational Surveys: How Informants Report About Interorganizational Relationships Joan M. Phillips Mendoza College of Business University of Notre Dame The author gratefully acknowledges the support provided by the Institute for the Study of Business Markets at Penn State.

Introduction n Conceptual Issues u Trend toward fewer but more purposive strategic partnerships n Substantive Issues u Growing interest among practitioners and scholars in understanding... F How these partnerships are formed F Reasons for failure F Determinants of successful relationships

Introduction n Methodological Issues u Variables related to successful relationships are unobservable u Obtaining valid measures of these latent constructs remains a challenge n Empirical work used survey methods u Application of cognitive psychology perspectives to improve survey data

Summary of Literature n Satisfaction in Vertical Marketing Systems u Growing interest in understanding interorganizational relations n Reliability and Validity of Organizational Informant Reports u Obtaining valid measures of organizational contructs remains a challenge n Cognitive Aspects of Proxy Reporting u Application of cognitive theories has improved the validity of self- and proxy-reports

Research Objectives n Understand how informants report about their organizations and its relationships u Cognitive processes used u Data sources used n Identify the factors associated with consensus among multiple informants n Recommend strategies to improve measures (i.e., questionnaire designs)

Sample n Organizations u 35 Industrial Manufacturers ($5 Mil.-$4 Bil.) F Automotive/Industrial Parts & Equipment[4] F Chemical Products[5] F Electrical/Electronic Components[4] F Heavy Equipment and Machinery[5] F Metal Products[6] F Machine Tools, Fasteners, & Abrasives[6] F Sheet Metals [5] n Informants u 109 Informants Interviewed - minimum of 3 per firm F General Management[27] F Sales[31] F Marketing[26] F Customer Service/Administrative[25]

Questionnaire n 6 Objective Items u % of Sales Volume from Distribution u % of Sales Volume from Distributor 1 u % of Sales Volume from Distributor 2 u Distributor 2’s Sales Volume Rank u Length of Relationship with Distributor 1 u Length of Relationship with Distributor 2 n 10 Subjective Items times 2 Distributors u Distributor’s Dependence u Manufacturer’s Dependence u Distributor’s Influence u Manufacturer’s Cooperation u Distributor’s Cooperation u Conflict u Distributor’s Communication u Trust u Distributor’s Performance u Manufacturer’s Satisfaction

Data Collection n Face-to-face interviews u Verbal protocol methodology F Cognitive “thinkalouds” u Audio recorded F Transcribed F Coded

Outcome Measures n Cognitive Processes u Anchoring u Decomposition u Calculation u Generalizing n Data Sources u Self/Job reference u Documents u Internal and external communication u Participation in event u Organizational cues n Consensus = Agreement among informants. u Variance for Subjective Questions u Relative Error (variance/mean 2 ) for Objective Questions

Consensus Hypotheses Supported n Similar Data Sources u Supported -- Use of similar data sources had a positive effect on informant consensus. n Perceived Question Difficulty u Supported for Objective Questions -- Perceived question difficulty has a negative effect on informant consensus for objective questions. n Importance of Target Firm u Supported for Objective Questions -- Importance of target firm had a positive effect on informant consensus

Consensus Hypotheses Not Supported n Similar Cognitive Processes u Not Supported -- Use of similar processes did not have an effect on informant consensus. n Use of Anchoring Processes u Not Supported -- Results for distributor 1 were mixed results for distributor 2 were in the opposite direction.

Data Source Hypotheses n Objective Questions u Supported -- Informants used self/job cues slightly more frequently than documents. u Supported -- The odds of using documents were greater for objective questions. n Subjective Questions u Supported -- Informants relied primarily on distributor organizational cues. u Not Supported -- The odds of using self/job cues, communication cues, and documents were greater for objective questions.

Anchoring Process Hypothesis n Use of Anchoring & Adjustment Processes u Supported -- When documents were not used, the odds of using an anchoring procedure for subjective questions was greater than for objective questions.

Contributions n Specify cognitive processes and information sources used by informants to report about interorganizational relationships n Understand how this is impacted by question type and informant characteristics n Identify the determinants of convergence among multiple informant reports n Recommend strategies for improving organizational surveys

Findings and Recommendations n FINDING u Informant agreement seems to be related to: F use of similar information sources u Informants use: F Organizational cues for subjective questions F Self/job cues and documents for objective data n RECOMMENDATION u Ask specific rather than general questions F e.g., Based on last month’s sales report... F e.g., Based on the written communication your firm received from this distributor last week...

Findings and Recommendations n FINDING u Use of an anchoring process is greater for subjective questions n RECOMMENDATION u Encourage use of same reference point F e.g., best distributor we have now, best ever, etc. u Define scale values for informants F Avoid Likert-type (e.g., SA/SD) scales

Findings and Recommendations n FINDING u Informants define organizational constructs differently (e.g., trust, performance, satisfaction, etc.) n RECOMMENDATION u Avoid single-item indicators of multi- dimensional constructs u Use global scales with caution

Generalizations n Can people really serve as informants? u But my experience matters most! n Do we know who our informants are? u Papa Bear, Momma Bear, & Baby Bear n Are we asking the right questions? u Which of your kids do you love the best? n What is most important? u Trust, Trust, Trust