Download presentation

Presentation is loading. Please wait.

Published byAllan Chow Modified over 2 years ago

1
Comparing Market Efficiency with Traditional and Non-Traditional Ratings Systems in ATP Tennis Dr Adrian Schembri Dr Anthony Bedford Bradley OBree Natalie Bressanutti RMIT Sports Statistics Research Group School of Mathematical and Geospatial Sciences RMIT University Melbourne, Australia www.rmit.edu.au/sportstats

2
Aims of the Presentation Structure of ATP tennis, rankings, and tournaments; Challenges associated with predicting outcomes of tennis matches; Utilising the SPARKS and Elo ratings to predict ATP tennis; Evaluate changes in market efficiency in tennis over the past eight years. RMIT University©2011 RMIT Sports Statistics 2

3
Background to ATP Tennis RMIT University©2011 RMIT Sports Statistics 3 ATP: Association of Tennis Professionals; Consists of 65 individual tournaments each year for men playing at the highest level; Additional: 178 tournaments played in the Challenger Tour; 534 tournaments played in Futures tennis.

4
ATP Tennis Rankings RMIT University©2011 RMIT Sports Statistics 4 Used to determine qualification for entry and seeding in all tournaments for both singles and doubles; The rankings period is always the past 52 weeks prior to the current week.

5
ATP Tennis Rankings – Sept 12th, 2011 RMIT University©2011 RMIT Sports Statistics 5

6
How Predictive are Tennis Rankings? Case Study RMIT University©2011 RMIT Sports Statistics 6 Australian Hardcourt Titles January, 1998 Adelaide, Surface – Hardcourt Lleyton Hewitt (AUS)Andre Agassi (USA) Age16 years27 years ATP Ranking55086 (6 th in Jan, 1999)

7
How Predictive are Tennis Rankings? Case Study RMIT University©2011 RMIT Sports Statistics 7 Robby GinepriRobin Soderling 6 - 4 7 - 5 Age16 years27 years Tourn SeedUnseeded1 Aircel Chennai Open January 4 - 10, 2010 Chennai, Surface – Hardcourt

8
Challenges Associated with Predicting Outcomes in ATP Tennis RMIT University©2011 RMIT Sports Statistics 8 Individual sport and therefore natural variation due to individual differences prior to and during a match; Constant variations in the quality of different players: Players climbing the rankings; Players dropping in the rankings; Players ranking remaining stagnant. The importance of different tournaments varies for each individual players.

9
Recent Papers on Predicting ATP Tennis and Evaluating Market Efficiency RMIT University©2011 RMIT Sports Statistics 9 Forrest and McHale (2007) reviewed the potential for long- shot bias in mens tennis; Klaassen and Magnus (2003) developed a probability- based model to evaluate the likelihood of a player winning a match, whilst Easton and Uylangco (2010) extended this to a point-by-point model; A range of probability-based models are available online, however these are typically volatile and reactive to events such as breaks in serve and each set result (e.g., www.strategicgames.com.au).

10
Aims of the Current Paper Evaluate the efficiency of various tennis betting markets over the past eight years; Compare the efficiency of these markets with traditional ratings systems such as Elo and a non-traditional ratings system such as SPARKS; Identify where inefficiencies in the market lie and the degree to which this has varied over time. RMIT University©2011 RMIT Sports Statistics 10

11
www.rmit.edu.au/sportstats Elo Ratings and the SPARKS Model

12
Introduction to Ratings Systems 12 RMIT University©2011 RMIT Sports Statistics Typically used to: Monitor the relative ranking of players with other players in the same league; Identify the probability of each team or player winning their next match. Have been developed in the context of individual (chess, tennis) or group based sports (e.g., AFL football, NBA); The initial ratings suggest which player is likely to win, with the difference between their old ratings being used to calculate a new rating after the match is played.

13
Introduction to SPARKS 13 RMIT University©2011 RMIT Sports Statistics Initially developed by Bedford and Clarke (2000) to provide an alternative to traditional ratings systems; Differ from Elo-type ratings systems as SPARKS considers the margin of the result; Has been recently utilised to evaluate other characteristics such as the travel effect in tennis (Bedford et al., 2011).

14
Introduction to SPARKS 14 RMIT University©2011 RMIT Sports Statistics where

15
Introduction to SPARKS 15 RMIT University©2011 RMIT Sports Statistics

16
SPARKS: Case Study 16 RMIT University©2011 RMIT Sports Statistics Robin Soderling (SWE)Ryan Harrison (USA) 6-2 6-4 Seeding1Qualifier Pre-Match Rating2986.3978.4 Expected Outcome20.1-20.1 Observed OutcomeWinLoss SPARKS246 SPARKS Difference18-18 Residuals-2.12.1 Post-Match Rating2975.9988.8

17
Longitudinal Examination of SPARKS 17 RMIT University©2011 RMIT Sports Statistics

18
Limitations of SPARKS: Case Study 18 RMIT University©2011 RMIT Sports Statistics PlayerSet 1Set 2Set 3Set 4CalculationSPARKS (Diff) Player 177721 + (3*6)39 (21) Player 266618 + (0*6)18 (21) PlayerSet 1Set 2Set 3Set 4CalculationSPARKS Player 1636621 + (3*6)39 (21) Player 2262212 + (1*6)18 (21) +

19
Limitations of SPARKS: Case Study 19 RMIT University©2011 RMIT Sports Statistics PlayerSet 1Set 2Set 3Set 4CalculationSPARKS (Diff) Player 177721 + (3*6)39 (21) Player 266618 + (0*6)18 (21) PlayerSet 1Set 2Set 3Set 4CalculationSPARKS Player 1636621 + (3*6)39 (21) Player 2262212 + (1*6)18 (21) Player 2 competitive in all three sets. +

20
Limitations of SPARKS: Case Study 20 RMIT University©2011 RMIT Sports Statistics PlayerSet 1Set 2Set 3Set 4CalculationSPARKS (Diff) Player 177721 + (3*6)39 (21) Player 266618 + (0*6)18 (21) PlayerSet 1Set 2Set 3Set 4CalculationSPARKS Player 1636621 + (3*6)39 (21) Player 2262212 + (1*6)18 (21) Player 2 competitive in all three sets. Player 2 competitive in 1 out of 4 sets.

21
Historical Results of the SPARKS Model 21 RMIT University©2011 RMIT Sports Statistics YearWin Prediction in all ATP Matches 2003.64 2004.64 2005.69 2006.67 2007.66 2008.67 2009.69 2010.72 The following table displays historical results of the raw SPARKS model over the past 8 years. +

22
Historical Results of the SPARKS Model 22 RMIT University©2011 RMIT Sports Statistics YearWin Prediction in all ATP Matches 2003.64 2004.64 2005.69 2006.67 2007.66 2008.67 2009.69 2010.72 The following table displays historical results of the raw SPARKS model over the past 8 years.

23
Banding of Probabilities 23 RMIT University©2011 RMIT Sports Statistics Lower Band Upper Band Midpoint 0.000.050.025 0.050.100.075 0.100.150.125 0.150.200.175 0.200.250.225 0.250.300.275 0.300.350.325 0.350.400.375 0.400.450.425 0.450.500.475 Probability banding is used primarily to determine whether a models predicted probability of a given result is accurate; Enables an assessment of whether the probability attributed to a given result is appropriate based on reviewing all results within the band; For example, if 200 matches within a given tennis season are within the.20 to.25 probability band, then between 20% and 25% (or approx 45 matches) of these matches should be won by the players in question.

24
Banding and the SPARKS Model 24 RMIT University©2011 RMIT Sports Statistics Lower Band Upper Band Midpoint 0.000.050.025 0.050.100.075 0.100.150.125 0.150.200.175 0.200.250.225 0.250.300.275 0.300.350.325 0.350.400.375 0.400.450.425 0.450.500.475 Lower Band Upper Band Midpoint 0.500.550.525 0.550.600.575 0.600.650.625 0.650.700.675 0.700.750.725 0.750.800.775 0.800.850.825 0.850.900.875 0.900.950.925 0.951.000.975 +

25
Banding and the SPARKS Model 25 RMIT University©2011 RMIT Sports Statistics Lower Band Upper Band Midpoint 0.000.050.025 0.050.100.075 0.100.150.125 0.150.200.175 0.200.250.225 0.250.300.275 0.300.350.325 0.350.400.375 0.400.450.425 0.450.500.475 Lower Band Upper Band Midpoint 0.500.550.525 0.550.600.575 0.600.650.625 0.650.700.675 0.700.750.725 0.750.800.775 0.800.850.825 0.850.900.875 0.900.950.925 0.951.000.975 Represent the underdog. Represent the favourite.

26
Banding and the SPARKS Model 26 RMIT University©2011 RMIT Sports Statistics

27
Banding and the SPARKS Model (2003-2010) 27 RMIT University©2011 RMIT Sports Statistics +

28
Banding and the SPARKS Model (2003-2010) 28 RMIT University©2011 RMIT Sports Statistics Under-estimates the probability of the under-dog winning. Over-estimates the probability of the favorite winning.

29
www.rmit.edu.au/sportstats Elo Ratings

30
Introduction to Elo Ratings 30 RMIT University©2011 RMIT Sports Statistics Elo ratings system developed by Árpád Élő to calculate relative skill levels of chess players where: R N = New rating R O = Old rating O = Observed Score E = Expected Score W = Multiplier (16 for masters, 32 for lesser players)

31
Probability Bands: Elo Ratings 31 RMIT University©2011 RMIT Sports Statistics +

32
Probability Bands: Elo Ratings 32 RMIT University©2011 RMIT Sports Statistics

33
33 RMIT University©2011 RMIT Sports Statistics Probability Bands: Elo Ratings (2003-2010)

34
34 RMIT University©2011 RMIT Sports Statistics Probability Bands: Elo Ratings (2003-2006) +

35
35 RMIT University©2011 RMIT Sports Statistics Probability Bands: Elo Ratings (2003-2006) High variability in the majority of probability bands during the burn-in period.

36
36 RMIT University©2011 RMIT Sports Statistics Probability Bands: Elo Ratings (2007-2010) +

37
37 RMIT University©2011 RMIT Sports Statistics Probability Bands: Elo Ratings (2007-2010)

38
Advantages and Shortcomings of SPARKS and Elo Ratings 38 RMIT University©2011 RMIT Sports Statistics SPARKS considers the margin of the result, often a difficult task in the context of tennis; Elo is only concerned with whether the player wins or loses, not the margin of victory in terms of the number of games or sets won; Elo provides a more efficient model in terms of probability banding, suggesting that evaluating the margin of matches may be misleading at times.

39
www.rmit.edu.au/sportstats Market Efficiency of ATP Tennis in Recent Years

40
ATP Betting Markets Used in the Current Analysis RMIT University©2011 RMIT Sports Statistics 40 MarketAbbreviation Bet 365B365 LuxbetLB ExpektEX Stan JamesSJ Pinnacle SportsPS Elo ratingsElo SPARKS

41
Overall Efficiency of Each Market between 2003 and 2010 RMIT University©2011 RMIT Sports Statistics 41 Market20032004200520062007200820092010Overall B365.71.67.70.71.72.71.70.703 LB.70.69.68.69.70.71.70.697 PS.71.65.70.68.72.70.696 SJ.69.70.67.73.69.70.71.696 EX.72.65.72.69.73.70.69.698 Elo.59.62.66.65.70.66.68.67.654 SPARKS.63.64.69.67.66.60.69.72.667 Overall.68.66.69.68.71.68.70.69 +

42
Overall Efficiency of Each Market between 2003 and 2010 RMIT University©2011 RMIT Sports Statistics 42 Market20032004200520062007200820092010Overall B365.71.67.70.71.72.71.70.703 LB.70.69.68.69.70.71.70.697 PS.71.65.70.68.72.70.696 SJ.69.70.67.73.69.70.71.696 EX.72.65.72.69.73.70.69.698 Elo.59.62.66.65.70.66.68.67.654 SPARKS.63.64.69.67.66.60.69.72.667 Overall.68.66.69.68.71.68.70.69

43
Overall Efficiency of Each Market between 2003 and 2010 RMIT University©2011 RMIT Sports Statistics 43 +

44
Overall Efficiency of Each Market between 2003 and 2010 RMIT University©2011 RMIT Sports Statistics 44 Heightened stability and efficiency across markets and seasons since 2008.

45
Converting Market Odds into a Probability RMIT University©2011 RMIT Sports Statistics 45 Novak DjokovicRafael Nadal Match Odds$1.63$2.25 Conversion1/1.631/2.25 Probability of Winning.61.44 2011 US Open Final

46
Accounting for the Over-Round RMIT University©2011 RMIT Sports Statistics 46 The sum of the probability-odds in any given sporting contest typically exceeds 1, to allow for the bookmaker to make a profit; The amount that this probability exceeds 1 is referred to as the over-round; For example, if the sum of probabilities for a given match is equal to 1.084, the over-round is equal to.084 or 8.4%

47
Accounting for the Over-Round RMIT University©2011 RMIT Sports Statistics 47 Novak DjokovicRafael Nadal Match Odds$1.63$2.25 Conversion1/1.631/2.25 Probability of Winning.61.44 Sum of Probabilities1.05 Over-Round5% 2011 US Open Final 6 – 2 6 – 4 6 – 7 6 – 1

48
Comparison of Over-Round Across Markets RMIT University©2011 RMIT Sports Statistics 48 +

49
Comparison of Over-Round Across Markets RMIT University©2011 RMIT Sports Statistics 49 Kruskal-Wallis test with follow-up Mann-Whitney U tests: Significant difference between all betting markets aside from Pinnacle Sports and Stan James.

50
Over-Round for Bet 365 (2003-2010) RMIT University©2011 RMIT Sports Statistics 50

51
Accounting for the Over-Round: Normalised Probabilities and Equal Distribution RMIT University©2011 RMIT Sports Statistics 51 Novak DjokovicRafael Nadal Match Odds$1.63$2.25 Raw Probability of Winning.61.44 Over-round.05 Normalisation.61/1.05.44/1.05 Normalised Probability of Winning.58.42 Equal Distribution.61 – (.05/2).44 – (.05/2) Equalised Probability of Winning.585.415 +

52
Accounting for the Over-Round: Normalised Probabilities and Equal Distribution RMIT University©2011 RMIT Sports Statistics 52 Novak DjokovicRafael Nadal Match Odds$1.63$2.25 Raw Probability of Winning.61.44 Over-round.05 Normalisation.61/1.05.44/1.05 Normalised Probability of Winning.58.42 Equal Distribution.61 – (.05/2).44 – (.05/2) Equalised Probability of Winning.585.415

53
Accounting for the Over-Round: Normalised Probabilities and Equal Distribution RMIT University©2011 RMIT Sports Statistics 53 Roger FedererBernard Tomic Match Odds$1.07$6.60 Raw Probability of Winning.93.15 Over-round.08 Normalisation.93/1.08.15/1.08 Normalised Probability of Winning.86.14 Equal Distribution.93 – (.08/2).15 – (.08/2) Equalised Probability of Winning.89.11 +

54
Accounting for the Over-Round: Normalised Probabilities and Equal Distribution RMIT University©2011 RMIT Sports Statistics 54 Roger FedererBernard Tomic Match Odds$1.07$6.60 Raw Probability of Winning.93.15 Over-round.08 Normalisation.93/1.08.15/1.08 Normalised Probability of Winning.86.14 Equal Distribution.93 – (.08/2).15 – (.08/2) Equalised Probability of Winning.89.11

55
Market Efficiency in ATP Tennis RMIT University©2011 RMIT Sports Statistics 55 +

56
Market Efficiency in ATP Tennis RMIT University©2011 RMIT Sports Statistics 56 SPARKS significantly less efficient when compared with the betting markets for all bands aside from.50 -.55.

57
Market Efficiency in ATP Tennis - Raw RMIT University©2011 RMIT Sports Statistics 57 +

58
Market Efficiency in ATP Tennis - Raw RMIT University©2011 RMIT Sports Statistics 58 General inefficiency across bands, likely due to no correction for the over-round.

59
Market Efficiency in ATP Tennis - Normalised RMIT University©2011 RMIT Sports Statistics 59

60
Market Efficiency in ATP Tennis – Equal Diff RMIT University©2011 RMIT Sports Statistics 60 +

61
Market Efficiency in ATP Tennis – Equal Diff RMIT University©2011 RMIT Sports Statistics 61 Relative consistency in efficiency and variability within each band across markets. +

62
Market Efficiency in ATP Tennis – Equal Diff RMIT University©2011 RMIT Sports Statistics 62 Evidence of longshot bias for the.25 to.30 band.

63
Market Efficiency in ATP Tennis: Bet365 RMIT University©2011 RMIT Sports Statistics 63 +

64
Market Efficiency in ATP Tennis: Bet365 RMIT University©2011 RMIT Sports Statistics 64

65
Longitudinal Changes in Market Efficiency RMIT University©2011 RMIT Sports Statistics 65 +

66
Longitudinal Changes in Market Efficiency RMIT University©2011 RMIT Sports Statistics 66 Few significant differences emerged when comparing efficiency across the bands over the past 8 years. Homogeneity of variance tests revealed significantly less variability across markets in recent years.

67
Most Efficient Year: 2007 RMIT University©2011 RMIT Sports Statistics 67 +

68
Most Efficient Year: 2007 RMIT University©2011 RMIT Sports Statistics 68

69
Least Efficient Year: 2004 RMIT University©2011 RMIT Sports Statistics 69 +

70
Least Efficient Year: 2004 RMIT University©2011 RMIT Sports Statistics 70

71
www.rmit.edu.au/sportstats Discussion of Findings

72
Psychological Player Considerations RMIT University©2011 RMIT Sports Statistics 72 Form of an individual player will affect the context and potential outcome of the entire match, as opposed to a team- based sport where individual players have less impact or can be substituted off if out of form. Micro-events within a match, at times, have an impact on the outcome of the match. Examples: Rain delays Injury Time outs Code violations

73
Shortcomings of the Current Analysis RMIT University©2011 RMIT Sports Statistics 73 A set multiplier of 6 was used for the SPARKS model based on the original SPARKS model published in 2000; Only a limited number of betting markets were incorporated, and therefore it was not possible to utilise Betfair data into the analysis; Differences in market efficiency and inefficiency were not evaluated at the surface level. This would be particularly interesting if evaluated for clay, given the volatility of player performance on clay when compared with other surfaces.

74
Future Work RMIT University©2011 RMIT Sports Statistics 74 Optimise the set multiplier of the SPARKS model; Develop a model that combines SPARKS and Elo ratings; Extend the current findings to incorporate womens tennis given that evidence has shown greater volatility in the womens game. Incorporate data on other potential predictors of tennis outcomes. Examples include: The set sequence of the match Surface Importance of the tournament (e.g., Grand slams)

75
Conclusions RMIT University©2011 RMIT Sports Statistics 75 Whilst considerable variability was evident during the 2003 – 2007 seasons, an increase in consistency across markets since 2008. Following a lengthy burn-in period of four years, the Elo model outperformed SPARKS and most betting markets across the majority of probability bands; Whilst not efficient in terms of probability banding, the SPARKS model was able to predict an equivalent proportion of winners to the betting markets, and outperformed some markets in recent years; A model that combines both Elo and SPARKS may yield the most efficient model.

76
Questions and Comments RMIT University©2011 RMIT Sports Statistics 76

77
RMIT University©2011 RMIT Sports Statistics 77

Similar presentations

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google