1. Managing Technical Talent: How to Find the Right Analyst for Your Problem Photo by mikebaird, www.flickr.com/photos/mikebaird Presentation to the Wolfram Data Summit Washington DC, Friday, Sept 09, 2011 nicholas.gruen@kaggle.com @nicholasgruen

2. genetic algorithms random forest Monte Carlo methods principal component analysis Kalman filter evolutionary fuzzy modelling neural networks logistic regression support vector machine decision trees ensemble methods adaBoost Bayesian networks Different users - different techniques.

3. “A discovery is... an accident meeting a prepared mind.” Albert Szent-Gyorgyi, 1937 Nobel Prize for Medicine ‣ Is the crown pure gold? ‣ We know its weight. ‣ How to measure its volume? Eureka!

4. 4

5. Finding the world’s most perfectly prepared mind

6. Our User Base

7. Competition Mechanics Competitions are judged on objective criteria

8. 1 23 Users create predictive models, submit these to Kaggle, and are scored on their accuracy. How Kaggle Works

9. Competitions are judged based on predictive accuracy

10. + Genetic marker 4 Genetic marker 1 + Genetic marker 3 + Genetic marker 2 Which HIV patients will be sicker next week?

11. HIV LoadStock PricesChess Ratings Scouring the world for the best analysts for a problem. Traffic flowGrant Forecasting Dr. Derek Gatherer UK John Blatz Baltimore Edmund & Adrian London & USA Jason Trigg Pennsylvania Chih-Li Sung & Roy Tseng Penghu & Taipei Jure Zbontar Ljubljana Thomas Mahony Canberra Emir Delic Australia Glen Maher Canberra Chris Raimondi Batimore Claudio Perlich USA Gzegorz Swiszcz Gera Edmund & Adrian London & USA Rajstennaj Barrabas USA Jason Trigg Pennsylvania Lee Baker Las Cruces, NM Cole Harris Texas Nan Zhou Pittsburgh Uri Blass Tel-Aviv Giuseppe Ragusa Rome Robert Warsaw Ivan Russian Federation Chris DuBois Portland Philipp Emanuel Widmann Heidelberg, DE Dr. Christopher Hefele, New York Jeremy Howard Chris Raimondi Baltimore Tim Salimans Erasmus U

12. Global competitions 1½ weeks 70.8% Competition closes 77% State of the art 70% Predicting HIV progression US$500

13. HIV LoadStock PricesChess Ratings Where’s Wally? Scouring the world for the best analysts for a problem. Traffic flowGrant Forecasting Dr. Derek Gatherer UK John Blatz Baltimore Edmund & Adrian London & USA Jason Trigg Pennsylvania Chih-Li Sung & Roy Tseng Penghu & Taipei Jure Zbontar Ljubljana Chris Raimondi Batimore Claudio Perlich USA Gzegorz Swiszcz Gera Edmund & Adrian London & USA Rajstennaj Barrabas USA Jason Trigg Pennsylvania Lee Baker Las Cruces, NM Cole Harris Texas Nan Zhou Pittsburgh Uri Blass Tel-Aviv Giuseppe Ragusa Rome Robert Warsaw Ivan Russian Federation Chris DuBois Portland Philipp Emanuel Widmann Heidelberg, DE Dr. Christopher Hefele, New York Chris Raimondi Baltimore

14. HIV LoadStock PricesChess Ratings Where’s Wally? Scouring the world for the best analysts for a problem. Traffic flowGrant Forecasting Dr. Derek Gatherer UK John Blatz Baltimore Edmund & Adrian London & USA Jason Trigg Pennsylvania Chih-Li Sung & Roy Tseng Penghu & Taipei Jure Zbontar Ljubljana Chris Raimondi Batimore Claudio Perlich USA Gzegorz Swiszcz Gera Edmund & Adrian London & USA Rajstennaj Barrabas USA Jason Trigg Pennsylvania Lee Baker Las Cruces, NM Cole Harris Texas Nan Zhou Pittsburgh Uri Blass Tel-Aviv Giuseppe Ragusa Rome Robert Warsaw Ivan Russian Federation Chris DuBois Portland Philipp Emanuel Widmann Heidelberg, DE Dr. Christopher Hefele, New York Tim Salimans Erasmus U R’dam

15. Martin O’Leary

16. “In less than a week … a PhD student in glaciology outperformed the state- of-the-art algorithms”

17. We could not be happier with the result. The Kaggle approach has set a new benchmark in Government for the development of successful predictive models, delivered quickly and very cost effectively. In particular, the flexibility of the winning predictive model will enable its application to other major transport routes to the CBD and allow for the addition of other factors such as weather and incident. Susan Calvert Director, Strategy and Project Delivery Unit Department Premier and Cabinet

18. A Few Kaggle Projects Take historical medical claims and predict who will go to hospital. This competition has a $3 million prize. Predict which editors will stop contributing New algorithm for chess ratings. Has wide gaming and ranking significance Detect driver drowsiness Predict the likelihood of claims given different vehicle models Predict successful grant applications Predict shoppers’ next visit to supermarket

19. User base: 14,107 registered data scientists

20. Forecast Error (MASE) Combination of world’s best models Aug 92 weeks later 1 month later Competition End This competition (to forecast tourism demand) used one of the most heavily studied sets of time series data. It had previously been modeled using the leading commercial software and academic algorithms. Competitors quickly surpassed world’s best practice and found the frontier of what’s possible. Frontier reached after all information is extracted from the dataset Kaggle Competition Results

21. HIV LoadStock PricesChess Ratings Where’s Wally? Scouring the world for the best analysts for a problem. Traffic flowGrant Forecasting Dr. Derek Gatherer UK John Blatz Baltimore Edmund & Adrian London & USA Jason Trigg Pennsylvania Chih-Li Sung & Roy Tseng Penghu & Taipei Jure Zbontar Ljubljana Chris Raimondi Batimore Claudio Perlich USA Gzegorz Swiszcz Gera Edmund & Adrian London & USA Rajstennaj Barrabas USA Jason Trigg Pennsylvania Lee Baker Las Cruces, NM Cole Harris Texas Nan Zhou Pittsburgh Uri Blass Tel-Aviv Giuseppe Ragusa Rome Robert Warsaw Ivan Russian Federation Chris DuBois Portland Philipp Emanuel Widmann Heidelberg, DE Dr. Christopher Hefele, New York Jeremy Howard

22. From generating value => Making money 1.Open Comps: Unleashing the power of Crowdsourcing $Commission, consulting and performance fees 2.Consulting partnerships $revenue share 3.The platform as marketplace for technical talent $revenue share

23. Our market Business analytics = $107 bil market Outsourced business analytics = $38b [IDC] Public and third sector Revenue forecasts Traffic forecasting Energy demand Predicting crime Tax/social security fraud Hospital casualty demand Identifying great Teachers Hospitals Private Sector Sales forecasts Credit scoring Stock picking Risk modelling and pricing Identifying fraud Identifying best practice Production management Inventory management Logistic optimisation

24. First mover advantages of internet platforms Clients Analysts

25. Kaggle not for profit Kaggle public good competitions

26. “I keep saying the sexy job in the next ten years will be statisticians. ” Hal Varian Google Chief Economist 2009 No matter who you are, most of the smartest people work for someone else. Bill Joye Founder, Sun Microsystems 2009

27. http://blog.costumemachine.com/tag/wheres-waldo-costume/ http://www.guardian.co.uk/world/picture/2010/jun/01/eyewitness-world-record-wally

28. Transforming the inefficient market for technical talent into the world’s largest meritocracy. nicholas.gruen@kaggle.com @nicholasgruen Wally Photos by William Murphy (Flickr: infomatique)

30. Who We Are Anthony Goldbloom CEO / Founder Econometrician @ the Australian Treasury & Reserve Bank of Australia Journalism Intern @ The Economist. Nicholas Gruen Chairman Chairman of the Australian Gov. 2.0 taskforce Jeff Moser CTO Developer @ Raytheon and widely read bloggerwidely read blogger Jeremy Howard Chief Scientist McKinsey and A.T. Kearney alumnus Founder of 2 successful startups: FastMail (exit to Opera) and Optimal Decisions Group (exit to Choicepoint)