1. Management field experiments Nick Bloom (Stanford and NBER) www.stanford.edu/~nbloom AOM, August 3 rd 2012

2. Management field experiments Review of current experimental literature Our projects in India and China Thoughts on running field experiments

3. The reason is running management experiments is expensive Surge in management experiments, but mainly in: (A) micro-firms (1 or 2 person) in developing countries (B) individual larger firms

4. Developing countries, micro-firm experiments Karlan & Valdivia in Peru; Bruhn, Karlan & Schoar in Mexico; Karlan & Udry in Ghana; McKenzie & Woodruff in Sri Lanka etc. Provide limited (≈50 hours) of basic trainings to small firms: accounting, marketing, pricing, strategy etc. Training is provided at random and data collected before & after So far finding not-much impact. I see two potential explanations -management does not matter in tiny firms, or -intervention is very poor quality

5. Single firms in developed countries, (1/2) Growing literature (surveyed in Bloom & Van Reenen, 2010, Handbook of Labour Economics) Classic examples include: Lazaer (2000, AER) on incentive pay at Safelite Glass, Shearer (2004, REStud) on tree planters and Hamilton et al (2003, JPE) on group incentives in factories

6. Single firms in developed countries (2/2) Recently Bandiera, Barankay and Rasul have an impressive set of papers. Run experiments on incentives for workers and managers, team selection, and task division on a fruit farm Introduce changes ½ way through season (using last season as the control), finding for example Worker incentive pay increases their performance, especially absolute (rather than relative) incentives Manager incentive pay improves team selection (less favoritism) and the effort they put into monitoring workers

7. Management field experiments Review of current experimental literature Our projects in India and China India China Thoughts on running field experiments

8. Does management matter? Evidence from India Nick Bloom (Stanford) Benn Eifert (Berkeley) Aprajit Mahajan (Stanford) David McKenzie (World Bank) John Roberts (Stanford GSB) http://www.stanford.edu/~nbloom/DMM.pdf

9. Management score Source: www.worldmanagementsurvey.comwww.worldmanagementsurvey.com 2.62.833.23.4 US Japan Germany Sweden Canada Australia UK Italy France New Zealand Mexico Poland Republic of Ireland Portugal Chile Argentina Greece Brazil China India One motivation for looking at management is that country management scores are correlated with GDP

10. Source: www.worldmanagementsurvey.comwww.worldmanagementsurvey.com Management score US (N=695 firms) India (N=620 firms) Density And firm management spreads look like TFP spreads

11. But does management cause any of these TFP differences between firms and countries? Massive literature of case-studies and surveys but no consensus Syverson (2011, JEL) “no potential driving factor of productivity has seen a higher ratio of speculation to empirical study”.

12. We ran an experiment on large firms to investigate the impact of modern management practices on TFP Experiment on 20 plants in large multi-plant firms (average 300 employees and $7m sales) near Mumbai making cotton fabric Randomized treatment plants got 5 months of management consulting intervention, controls got 1 month Consulting was on 38 specific practices tied to factory operations, quality and inventory control Collect weekly performance data from 2008 to August 2010, and long-run size and management data from 2008 to 2011

13. Exhibit 1: Plants are large compounds, often containing several buildings.

14. Fabric weaving Exhibit 2: Plants operate continuously making cotton fabric from yarn

15. Exhibit 3: Many parts of these Indian plants were dirty and unsafe Garbage outside the plantGarbage inside a plant Chemicals without any coveringFlammable garbage in a plant

16. Exhibit 4: The plant floors were often disorganized and aisles blocked Instrument not removed after use, blocking hallway. Tools left on the floor after use Dirty and poorly maintained machines Old warp beam, chairs and a desk obstructing the plant floor

17. Yarn piled up so high and deep that access to back sacks is almost impossible Exhibit 5: The inventory rooms had months of excess yarn, often without any formal storage system or protection from damp or crushing Different types and colors of yarn lying mixed Yarn without labeling, order or damp protection A crushed yarn cone, which is unusable as it leads to irregular yarn tension

18. Intervention aimed to improve 38 core textile management practices in 6 areas – for example Targeted practices in 6 areas: operations, quality, inventory, HR and sales & orders

19. Months after the diagnostic phase.2.3.4.5.6 -10-8-6-4-2024681012 Adoption of the 38 management practices rose Treatment plants Control plants Share of 38 practices adopted Non-experimental plants in treatment firms Months after the start of the diagnostic phase

20. In terms of performance looked at four outcomes we have weekly data for Quality Inventory Output Productivity (defined as: Log(VA) – 0.42*log(K) – 0.58*log(L)) Use weekly data from March 2008 until August 2010 (after which some firms started upgrading to Jacquard looms)

21. Poor quality meant 19% of labor went on repairs Workers spread cloth over lighted plates to spot defectsLarge room full of repair workers (the day shift) Defects lead to about 5% of cloth being scrappedDefects are repaired by hand or cut out from cloth

22. 22 Previously mending was recorded only to cross- check against customers’ claims for rebates

23. Now mending is recorded daily in a standard format, so it can analyzed by loom, shift, design & weaver

24. The quality data is now collated and analyzed as part of the new daily production meetings Plant managers meet with heads of departments for quality, inventory, weaving, maintenance, warping etc.

25. Quality improved significantly in treatment plants Control plants Treatment plants Weeks after the start of the experiment Quality defects index (higher score=lower quality) Note: solid lines are point estimates, dashed lines are 95% confidence intervals

26. Stock is organized, labeled, and entered into the computer with details of the type, age and location. Organizing and racking inventory enables firms to substantially reduce capital stock

27. Many treated firms have also introduced basic initiatives to organize the plant floor Marking out the area around the model machine Snag tagging to identify the abnormalities

28. 28 Spare parts were also organized, reducing downtime (parts can be found quickly) Nuts & bolts Tools Spare parts

29. Production data is now collected in a standardized format, for discussion in the daily meetings Before (not standardized, on loose pieces of paper) After (standardized, so easy to enter daily into a computer)

30. TFP rose in treatment plants vs controls Control plants Treatment plants Weeks after the start of the experiment Total factor productivity Note: solid lines are point estimates, dashed lines are 95% confidence intervals

31. Why do badly managed firms exist? Competition heavily restricted by trade restrictions, the difficulty of new firms entering (finance is hard to raise), and the difficulty of good current firms expanding (limited by family size) Information is limited: firms either not aware of modern practices or simply do not believe they matter (“not worth the it”)

32. Management field experiments Review of current experimental literature Our projects in India and China India China Thoughts on running field experiments

33. Does Working from Home Work? Evidence from a Chinese Experiment Nick Bloom (Stanford) James Liang (Ctrip & Stanford) John Roberts (Stanford) Jenny Ying (Stanford) http://www.stanford.edu/~nbloom/WFH.pdf

34. 34 CTrip, a large NASDAQ listed Chinese multinational, wondered about introducing working from home CTrip, China’s largest travel-agent (13,000 employees, and $5bn value on NASDAQ). James Liang is the co-founder, first CEO and Chairman

35. The randomization into working from home was done publicly and also shown on the firm intranet Open lottery over even/odd treatment Working at Home Working at home Working at Home

36. Experiment yielded four learnings for the firm: (1) Working-from-home works (on average) Normalized calls per week Before the experimentDuring the experiment Control Treatment

37. Experiment yielded four learnings for the firm: (2) Better & worse workers both improve when WFH Normalized calls per week: difference between home and work Before experimentDuring experiment

38. Experiment yielded four learnings for the firm: (3) Selection: Worker choice increases WFH impact Normalized calls per week: difference between home and work During the experiment After the experiment (roll-out) Before the experiment

39. Experiment yielded four learnings for the firm: 4) Employees value WFH as attrition down 50% Note: average daily commute is 1.41 hours and cost $0.96

40. Experiment so successful that CTrip is rapidly rolling out WFH across the firm Profit increase per employee WFH about $2,000 per year: –Rent: $1,200 per year –Retention: $400 per year –Labor costs: $300 per year

41. Management field experiments Review of current experimental literature Our projects in India and China India China Thoughts on running field experiments

42. Thoughts on experiments 1.Expensive and hard, but worthwhile for the right question 2.Risky for junior faculty as can take many years 3.Think about both measure and intervention – both can be tough (in India measuring the control firms was tough) 4.Works best as a team effort – design, funding and execution all best as joint production 5.Running pilots and spending time on the ground invaluable for effective operation, analysis and presentation

43. Management field experiments Nick Bloom (Stanford and NBER) www.stanford.edu/~nbloom AOM, August 2012