1. The Economist as Engineer From design to adoption of matching- market clearinghouses: Theory, field data, experiments

2. 2 Matching markets Labor markets: match workers and jobs School choice systems: match students and schools Organ transplant rules and procedures: match patients and organs/donors My colleagues and I have had the opportunity to take a number of such design projects from initial conception to adoption and operation. The failures that sometimes afflict such markets give a new view of what obstacles matching markets have to overcome to create efficient matchings.

3. 3 Market design is field that requires eclectic methods Examination of contemporary and historical field data –Evolution of market rules and outcomes over time Game theory –Provides the tools to study incentives –Diagnose market failures –Design appropriate remedies Experiments –Allow hypotheses to be investigated that can’t be distinguished on the basis of the field data.

4. 4 Experiments play a role In diagnosing and understanding market failures, and successes In designing new markets In communicating results to policy makers

5. 5 What must matching markets do to operate efficiently? Whether centralized or decentralized, They have to –provide thickness –overcome congestion –make it safe for participants to reveal enough of their preferences to find an efficient matching We can see each of these problems in the history of the labor market for new doctors.

6. 6 Matching doctors to first positions in U.S. and Canada The redesign in 1995 of the –U.S. National Resident Matching Program (NRMP) (approx. 17,000 U.S. medical grads, 23,000 positions, 500 couples, 25,000 positions) –Canadian Resident Matching Service (CaRMS) (1,400 Canadian medical grads, 41 couples, 1,500 positions in 2005) The two markets are coordinated: CaRMS operates two ‘iterations’ typically right before and after the NRMP The redesign in 2005 of the fellowship matches for more senior physicians –The Gastroenterology fellowship match

7. 7 Centralized clearinghouses The NRMP and the CaRMS use a recently redesigned centralized clearinghouse –Roth, A.E. and E. Peranson, "The Redesign of the Matching Market for American Physicians: Some Engineering Aspects of Economic Design, American Economic Review, 89, 4, September, 1999, 748-780. In the clearinghouse, doctors and residency programs each fill out Rank Order Lists of preferences, and are matched by a computerized algorithm.

8. 8 Background to redesign of the medical clearinghouses 1900-1945 UNRAVELLING OF APPOINTMENT DATES 1945-1950 CHAOTIC RECONTRACTING--Congestion 1950-197x HIGH RATES OF ORDERLY PARTICIPATION ( 95%) in centralized clearinghouse 197x-198x DECLINING RATES OF PARTICIPATION (85%) particularly among the growing number of MARRIED COUPLES 1995-98Market experienced a crisis of confidence with fears of substantial decline in orderly participation; –Design effort commissioned—to design and compare alternative matching algorithms capable of handling modern requirements: couples, specialty positions, etc. –Roth-Peranson clearinghouse algorithm adopted, and employed

9. 9 Stages and transitions observed in various markets Stage 1: UNRAVELING Offers are early, dispersed in time, exploding…no thick market Stage 2: UNIFORM DATES ENFORCED Deadlines, congestion Stage 3: CENTRALIZED MARKET CLEARING PROCEDURES

10. 10 A simple centralized clearinghouse: deferred acceptance (has been discovered independently in many markets) Applicants and firms interview, and then submit rank order lists of one another (applicants are substitutes) Applicant-proposing deferred acceptance (Gale-Shapley): –Each applicant applies to his/her highest ranked firm –Each firm rejects unacceptable applicants and “holds” its highest ranked applications (up to the number of positions it has, and rejects the rest)… –At any stage at which an applicant has been rejected, he/she applies to her next most preferred firm –Each firm holds its most preferred set of applications and rejects the rest. –The algorithm stops when no rejections are issued, and each firm is matched to the applicants it is holding. Theorems: The outcome is Pareto efficient and stable, there does not exist a firm-applicant pair that would prefer to be matched to one another. And the procedure gives applicants straightforward (dominant strategy) incentives to state their true preferences.

11. 11 What makes a clearinghouse successful or unsuccessful? Clearinghouses tend to be built in markets suffering from failures related to lack of thickness (unraveling), congestion, or incentives. –A well designed clearinghouse is one way to solve all these problem. A matching is “stable” if there aren’t a doctor and residency program, not matched to each other, who would both prefer to be. Hypothesis: successful clearinghouses produce stable matchings. How to test this?

12. 12 MarketStableStill in use (halted unraveling) NRMP yesyes (new design in ’98) Edinburgh ('69)yesyes Cardiffyesyes Birminghamnono Edinburgh ('67)nono Newcastlenono Sheffieldnono Cambridgenoyes London Hospitalnoyes Medical Specialtiesyesyes (~30 markets, 1 failure) Canadian Lawyersyesyes (Alberta, no BC, Ontario) Dental Residenciesyes yes (5 ) (no 2) Osteopaths (< '94)nono Osteopaths (> '94)yes yes Pharmacistsyes yes Reform rabbisyes (first used in ‘97-98)yes Clinical psychyes (first used in ‘99)yes So stability looks like an important feature of a centralized labor market clearinghouse.

13. 13 The need for experiments How to know if the difference between stable and unstable matching mechanisms is the key to success? –There are other differences between e.g. Edinburgh and Newcastle The policy question is whether the new clearinghouse needs to produce stable matchings (along with all the other things it needs to do like handle couples, etc. ) –E.g. rural hospital question…

14. 14 A matching experiment Kagel, John H. and A.E. Roth, "The dynamics of reorganization in matching markets: A laboratory experiment motivated by a natural experiment," Quarterly Journal of Economics, February, 2000, 201-35.) 6 firms, 6 workers (half "High productivity" half "low productivity") It is worth $15 plus or minus at most 1 to match to a high It is worth $5 plus or minus at most 1 to match to a low There are three periods in which matches can be made:-2, -1, 0. Your payoff is the value of your match, minus $2 if made in period -2, minus $1 if made in period -1 Decentralized match technology: firms may make one offer at any period if they are not already matched. Workers may accept at most one offer. Each participant learns only of his own offers and responses until the end of period 0. After experiencing ten decentralized games, a centralized matching technology was introduced for period 0 (periods -2 and -1 were organized as before). Centralized matching technology: participants who are still unmatched at period 0 submit rank order preference lists, and are matched by a centralized matching algorithm. Experimental variable: Newcastle (unstable) or Edinburgh (stable) algorithm.

15. 15

16. 16 Firms keep making early offers, but workers stop accepting them

17. 17 What do the experiments add to the field data? 1.They show that stable versus unstable clearinghouse organization is important, in a controlled environment in which this is the only difference. 2.Demonstrates robustness: we’ve now seen the same phenomena in markets ranging in size from 20,000+ participants, to hundreds, to half a dozen. 3.Suggests that the asymmetry between firms and workers—and the ability to reject (or delay) unwanted offers—plays a critical role. Point 3 looms large in the U.S. market for law clerks for Federal appeals court judges. (Law students find it hard to say no to these guys… )

18. 18 The Roth-Peranson algorithm is also used in other market clearinghouses Postdoctoral Dental Residencies in the United States Psychology Internships in the United States and Canada Neuropsychology Residencies in the United States and Canada Osteopathic Internships in the United States Pharmacy Practice Residencies in the United States Articling Positions with Law Firms in Alberta, Canada

19. 19 NRMP also runs Fellowship matches through the Specialties Matching Service (SMS) Abdominal Transplant Surgery (2005) Child & Adolescent Psychiatry (1995) Colon & Rectal Surgery (1984) Combined Musculoskeletal Matching Program (CMMP) –Hand Surgery (1990) Medical Specialties Matching Program (MSMP) –Cardiovascular Disease (1986) –Gastroenterology (1986-1999; rejoined in 2006) –Hematology (2006) –Hematology/Oncology (2006) –Infectious Disease (1986-1990; rejoined in 1994) –Oncology (2006) –Pulmonary and Critical Medicine (1986) –Rheumatology (2005) Minimally Invasive and Gastrointestinal Surgery (2003) Obstetrics/Gynecology –Reproductive Endocrinology (1991) –Gynecologic Oncology (1993) –Maternal-Fetal Medicine (1994) –Female Pelvic Medicine & Reconstructive Surgery (2001) Ophthalmic Plastic & Reconstructive Surgery (1991) Pediatric Cardiology (1999) Pediatric Critical Care Medicine (2000) Pediatric Emergency Medicine (1994) Pediatric Hematology/Oncology (2001) Pediatric Rheumatology (2004) Pediatric Surgery (1992) Primary Care Sports Medicine (1994) Radiology –Interventional Radiology (2002) –Neuroradiology (2001) –Pediatric Radiology (2003) Surgical Critical Care (2004) Thoracic Surgery (1988) Vascular Surgery (1988)

20. The market for gastroenterologists Why did the market for gastroenterologists collapse? With what consequences? How to get it back on its feet? –Science/persuasion/politics… Experiments play a role in each of these, and complement other kinds of economic investigation (in this case theory, surveys, analysis of historical data...)

21. 21 Niederle, Muriel and Alvin E. Roth, “Relationship Between Wages and Presence of a Match in Medical Fellowships,” JAMA. Journal of the American Medical Association, vol. 290, No. 9, September 3, 2003, 1153-1154. Niederle, Muriel and Alvin E. Roth, “Unraveling reduces mobility in a labor market: Gastroenterology with and without a centralized match,” Journal of Political Economy, vol. 111, no. 6, December 2003, 1342-1352. Niederle, Muriel and Alvin E. Roth, “The Gastroenterology Fellowship Market: Should there be a Match?,” American Economic Review, Papers and Proceedings, 95,2, May, 2005, 372-375. C. Nicholas McKinney, Muriel Niederle, and Alvin E. Roth, “The collapse of a medical labor clearinghouse (and why such failures are rare),” American Economic Review, 95, 3, June, 2005,878-889. Niederle, Muriel and Alvin E. Roth, ““Making Markets Thick: How Norms Governing Exploding Offers Affect Market Performance,”,” working paper. Niederle, Muriel and Alvin E. Roth, “The Gastroenterology Fellowship Match: How it failed, and why it could succeed once again,” Gastroenterology, 127, 2 August 2004, 658-666. Niederle, Muriel, and Alvin E. Roth, “Re-starting the Gastroenterology Match,” letter, American Journal of Gastroenterology, vol. 100, no. 5, May, 2005, 1202-1203. Niederle, Muriel, Deborah D. Proctor, and Alvin E. Roth, “What will be needed for the new GI fellowship match to succeed?” Gastroenterology, January, 2006, 130, 218-224.

22. 22 Early History of the Gastro Market: Unraveling and Congestion Gastroenterology (2-3 years): subspecialty of Internal Medicine (3 years). Before 1986: Decentralized Process. –Residents apply for fellowship positions, receive interviews, and offers from hospitals. –However, they experienced market failure in terms of unraveling (early, dispersed, exploding offers). –Interim attempts to solve the problem: “Setting guidelines for interviewing candidates and negotiating positions was tried, and it was unsuccessful.” (Congestion)

23. 23 1986-~1996: The Match (MSMP) Applicants (and hospitals) submit rank order lists over hospitals (and applicants). The MSMP uses a version of a hospital proposing deferred acceptance algorithm, which yields stable outcomes (since ’98 they use the Roth-Peranson algorithm). When such a match has a high participation rate, it simultaneously solves the timing, congestion, and incentives problems.

24. 24 Gastro and health care reform Gastroenterology, in ’93-94 (the midst of health care reform): Manpower analysis. Meyer et al 1996: JAMA: –25-50% reduction of GI fellows (over the next 5 years) endorsed as goal by GI leadership council. –Starting with summer 1996: 3 years of GI fellowship required for board certification eligibility

25. 25 The demise of the Match Within 4 years the Match collapsed. After 1997 the match was effectively abandoned, and in 2000 it was formally abandoned YearPercent withdrawnPosts in MatchPercent Matched Applicants per Position ’92 --37796.61.75 ’93 - 6.7399941.6 ’94 --369931.6 ’95 433788.71.3 ’96 4.829874.80.9 ’97 16.1213851.1 ’98 44.39977.81.5 ‘99 6014--

26. 26 What have been the consequences of the demise of the match? The market has become –Less thick in time and space Earlier and more dispersed in time Less mobility –No changes in wages

27. 27 Timing of Interviews: Unraveling

28. 28 Dates during which fellowship programs were making offers. Each program is represented by one of the horizontal lines, indicating the (maximal) dates during which it could have had outstanding offers (2005 survey data, n=44). (As of November 15, 11 (27%) programs had already finished making offers, 12 (25%) had not yet started, and 21 (48%) were in the midst.)

29. 29 Effects of a Match: Mobility In the absence of a match, the market was more local, less national than when the match was operating. The decentralized market unraveled so that offers were earlier and more dispersed than during the years of the match. No effect on wages (Niederle and Roth, JAMA 2003.)

30. 30 Why did the Gastro Match break down? And why are such failures so rare? Some Hypotheses A centralized Match only works when there are more applicants than positions. –There may not be enough “high quality” applicants to fill high quality positions. The match failed because of the shock that reduced the demand of positions below the supply. The match failed because, when this shock occurred, fellowship programs were aware of it, but applicants were not. –Programs could update their priors when they saw how many applications they received.

31. 31 How to sort among these hypotheses? Each of them is consistent with the historical data. And, since stable match failures are rare, there isn’t a good possibility of a cross- market comparison. But in the laboratory, we can shock a market in different ways, and try to make a stable matching mechanism fail.

32. 32 A simple experimental environment 2 types of firms and workers; "High" and "low” productivity Matching to a High type is worth 150 points + private value: in [-10,10]). To a low type is worth 50 (+ private value) There are three periods in which to match: -2, -1, 0. Your payoff is the value of your match, minus 20 points if made in period -2, minus 10 points if made in period -1 Decentralized match technology: firms may make one offer at any period if they are not already matched. Workers may accept at most one offer. Each participant learns only of his own offers and responses until the end of period 0. (no period 0 eq.) After experiencing ten decentralized games, a centralized matching technology was introduced for period 0 (periods -2 and -1 were organized as before). Centralized matching technology: participants who are still unmatched at period 0 submit rank order preference lists, and are matched by a centralized (stable) matching algorithm.

33. 33 Experimental conditions Different supply and demand (between subjects variable) Shocks that change the long side of the market (within subject variable) Decentralized and Centralized matching (within subject) Different information conditions (symmetric or asymmetric between firms and workers) (between subject variable)

34. 34 Experimental Results Simple demand/supply imbalances have no effect. Shocks disturb the match, especially when applicants are not aware of them. Gradual adoption of the match…

35. 35 Full Information

36. 36 After the shock firms start making more early offers to workers. Workers: –Before the shock: high type workers eagerly accept early (-1) high offers. –After the shock: firms are in excess supply, workers have no big incentive to accept early offers. Workers only know this when they are informed of the shock, and indeed in that case accept early offers with much lower propensity. The ability of applicants to reject early offers that prevents unraveling. No other specialty experienced such shocks: Cardiovascular: from 1990 to 1998 the ratio of applicants to positions offered varied from a high of 1.6 to a low of 1.3. Pulmonary disease those ratios varied from a high of 1.5 to a low of 1.1, For Infectious disease (from 1994 to 1998) those ratios vary from a low of.68 to a high of.92.

37. 37 Gastro market—preliminary advice to the Gastroenterologists It appears that the collapse of the match was due to a shock, and the match could succeed if re-started. This would increase the mobility of GI fellows. Not all GI programs would do better: some small programs enjoy having a captive market (particularly if they are at a hospital with a strong internal medicine residency program)

38. 38 Transition to a later market In May 2005, the American Gastroenterological Association (AGA), the American College of Gastroenterology (ACG), the American Society for Gastrointestinal Endoscopy (ASGE) and the American Association for the Study of Liver Diseases (AASLD) decided to reintroduce a GI fellowship match, starting in 2006, for positions beginning in July 2007. How to manage the transition? –Rates of participation Concern among programs about whether their chief competitors will participate. –Change of dates to June 2006 (from as early as July 2005) –How to reassure programs that other programs will wait for Match?

39. 39 Market Culture (Niederle and Roth 2005) Different markets have different cultures: e.g. rules, norms, and expectations about how and when offers will be made, accepted, rejected. –In some markets, an exploding offer is easy to enforce and rely on, in other markets, making such offers would be frowned on, and other market participants wouldn’t help in enforcing the resulting contract. Many markets have organizations that try to change and influence these norms in order to promote more orderly markets and control the timing of the markets.

40. 40 Organizations concerned with how and when offers are made, accepted, rejected Council of Graduate Schools (CGS): graduate admissions, National Association for College Admission Counseling (NCAC): undergraduate admissions, National Resident Matching Program (NRMP): entry level medical residencies, (also Canadian Resident Matching Service – CaRMS – and various regional matches in Britain) Specialty Matching Services (SMS): advanced medical residencies and fellowships, Association of Psychology Postdoctoral and Internship Centers (APPIC): clinical psychology positions, National Association for Law Placement (NALP) for positions in law firms, Judicial Conference of the United States and various ad hoc committees of judges for federal judicial clerkships, Provincial Law Societies in Canada. National Association of Colleges and Employers (NACE) for US college undergraduates, NCAA: formerly for postseason college football bowls, now regulated by the Bowl Championship series (BCS), National Panhellenic Conference for sorority matching The Japan Federation of Employers’ Associations (Nikkeiren) for japanese university graduates National Collegiate Athletic Association (NCAA) for recruitment of college athletes, and various drafts…

41. 41 Which of these markets are unraveled? Of these markets, it appears that markets in which transactions are made at early, uncoordinated times are markets in which there are both –Exploding offers –Binding commitments Example: Graduate admissions has a nicely designed set of market rules, sent to all admitted students, that suppresses exploding offers (in part by facilitating “reneges” of premature acceptances).

42. 42 “Students are under no obligation to respond to offers of financial support prior to April 15; earlier deadlines for acceptance of such offers violate the intent of this Resolution. In those instances in which a student accepts an offer before April 15, and subsequently desires to withdraw that acceptance, the student may submit in writing a resignation of the appointment at any time through April 15.

43. 43 An experiment allows us to view different offer regimes in a controlled environment 5 firms, 6 applicants, 9 periods. In each period, a firm may make an offer to at most one applicant. Firms make offers, applicants decide upon the offers they receive. Firms and applicants are assigned “qualities.” If firm of quality x hires an applicant of quality y, both firm and applicant will receive a payoff of xy points each. Firms’ qualities are simply their assigned participant number, 1,2,3,4,5. Uncertainty about applicants’ quality is only resolved over time: In periods 1, 4 and 7, each applicant receives a “signal,” an integer between 1 and 10, each equally likely. In period 7, the relative ranking of the sum of the 3 signals determines the applicants’ quality. The applicant with the highest sum of 3 signals has a quality of 6… Ties are broken randomly. Firms see all signals, applicants see only their own signals (as they become available over time) and their ranking in period 7. (Note that in this experiment the cost of early matching is bad matches due to uncertainty about quality.)

44. 44 Experimental treatments: 3 “market cultures” Exploding offers: Firms can make exploding offers and acceptances are binding. Renege: Firms can make exploding offers, but applicants can renege on their acceptance, for a small fee (1 point). Open offers: Firms can only make open offers. (Many equilibria: One in which all matches are agreed upon inefficiently early. All environments have a perfect equilibrium with efficient late matching. But the late matching equilibrium is more fragile when offers are exploding and acceptances are binding.)

45. 45 Timing of final offers (offers that were accepted and not reneged upon for the renege treatment) in terms of the number of signals that were observed.

46. 46 In last five periods…

47. 47 Not everyone loses…(so it can be hard to get consensus on reversing unraveling:)

48. 48 So…How should Gastro transition from early to late?

49. 49 Proposed AGA resolution 2. Acceptance of Offers - Before, Through and After the Match: Gastroenterology fellowship applicants are under no obligation to accept an offer prior to the Match. Furthermore, applicants are not bound by the acceptance of an offer prior to the Match. … 3. Eligibility for the Match: All applicants are eligible to participate in the Match, whether or not they have accepted an offer prior to the Match. When an applicant is pressured by a program into accepting an offer prior to the Match, he may accept that offer and still remain eligible to participate in the Match. Since any agreement reached through the Match is binding, applicants should only submit to the Match a Rank Order List of programs they definitely prefer to any offer accepted before the Match. Applicants who have accepted an offer before the Match, and received a position in the Match, must announce their resignation to the program that made the early offer on the same day that they learn the Match result. … Applicants who accepted an early offer, but were not matched, must honor their original agreement. Applicants who receive several early (pre-Match) offers to which they are pressed to reply before the Match, can at any time accept the one they prefer, and should contact the programs whose offers they reject, whether they have accepted them beforehand, or were simply considering the offer. While applicants maintain their eligibility for the Match, even if they are pressured to accept offers before the match, such early offers violate the intent of this resolution. When applicants and programs are each others’ first choice, the Match will automatically match them when they submit those preferences on their Rank Order Lists. When all participants comply with the spirit of this resolution, all positions will be offered through the Match, or after it.

50. 50 Resolution Regarding Gastroenterology Fellowship Applicants, Including all Research and Clinical Fellowship Applicants and Positions This resolution concerns the conditions surrounding gastroenterology fellowship offers to applicants, acceptance by applicants of such offers, and participation by applicants and programs in the gastroenterology fellowship Match. The general spirit of this resolution is that each applicant should have an opportunity to consider all programs before making a decision and be able to participate in the Match. … The intention of this resolution is to ensure uniformity so that everyone participates fairly and to establish the principle that all positions should be filled through the Match or after Match Day. It therefore seeks to create rules that give both programs and applicants the confidence that applicants and positions will remain available to be filled through the Match and not withdrawn in advance of it. This resolution addresses the issue that some applicants may be persuaded or coerced to make commitments prior to, or outside of, the Match. Early offers and acceptances, and offers outside of the Match, are violations of the rules and of this resolution and are not condoned. Any applicant may participate in the matching process by registering for the Match to interview and consider match-participating programs; however, an applicant who accepts a position prior to, or outside of, the Match must comply with the National Resident Matching Program/Specialty Matching Services (NRMP/SMS) Match Participation Agreement by either resigning the accepted position if he/she wishes to submit a rank order list of programs or by withdrawing from the Match prior to the rank order list certification deadline, which is the first week in June. In addition, no program may withdraw a position from the Match after the quota change deadline to offer that position outside the matching process. … The spirit of this resolution is to make it unprofitable for program directors to press applicants to accept early offers, and to give applicants an opportunity to consider all offers as well as to provide uniform and widely acceptable rules that protect both applicants and fellowship programs.

51. 51 June 2006 The Gastro match for 2007 fellows was held June 21, 2006. Program Statistics Number % –151 eligible programs (+3 military) –Enrolled Programs 14797% –Withdrawn Programs 13 9% –Active Programs 134 –Programs Filled 129 96% –Programs Unfilled 5 4% –Active Positions 283 –Positions Filled 276 98% –Positions Unfilled 7 2%

52. 52 Conclusions: Markets don’t always spring up like weeds, some of them are hothouse orchids that need care and attention.

53. 53 Experiments have multiple roles to play They allow us to investigate questions that the field data cannot answer –E.g. why did the GI match fail in ’96? They allow us to investigate hypotheses suggested by the field data –E.g. is stability the key to understanding the successful and failed British medical matches? –E.g. does the ability to renege on early acceptances reduce exploding offers and unraveling? They play a role in the considerable amounts of discovery, demonstration, and persuasion that are necessary to coordinate market participants to move from one equilibrium to a better one.

54. 54 The Economist as Engineer We need to understand how markets work in enough detail so we can fix them when they’re broken. And we’ll need to build an engineering- oriented economics literature if we’re going to get better at this. –Roth, Alvin E. “The Economist as Engineer: Game Theory, Experimental Economics and Computation as Tools of Design Economics,” Fisher Schultz lecture, Econometrica, 70, 4, July 2002, 1341-1378. –Market Design, Ec2056, Spring 2007

55. 55 Reminder: Next Week December 8, we meet in the lab, in the basement of Shad, for a Ztree class, taught by Ben Greiner.