1. Gürdal Ertek ertekg@sabanciuniv.edu
Analytical Benchmarking Meets Data Mining: The SmartDEA Framework, SmartDEA Software, and Case Studies for Industry
Gürdal Ertek
Invited Seminar at A*Star SIMTECH, Singapore, August 2, 2013, Friday

2. Istanbul, Turkey
Singapore

3. Young, high-profile private University Outskirts of Istanbul, Turkey
First students accepted in 1999

4. Established by the Sabanci Foundation

5. Sabancı Group

6. Sabancı Family: Sakıp Sabancı, Güler Sabancı, 200+

7. ~3000 undergrad & ~500 grad students

8. Highest research income per faculty member among Turkish universities

9. Young, high-profile private University
Established by the Sabanci Foundation
Sabancı Group
Sabancı Family: Sakıp Sabancı, Güler Sabancı, 200+
First students accepted in 1999
~3000 undergrad & ~500 grad students
Highest research income per faculty member

10. Dr. Gürdal Ertek
Assistant Professor at Sabancı University, Istanbul, Turkey, since 2002
Ph.D. from School of Industrial and Systems Georgia Institute of Technology, Atlanta, GA, USA
Research areas include
warehousing & material handling
data visualization & data mining

11. Analytical Benchmarking Meets Data Mining: The SmartDEA Framework, SmartDEA Software, and Case Studies for Industry
Gürdal Ertek
Invited Seminar at A*Star SIMTECH, Singapore, August 2, 2013, Friday

12. Motivation Analytical Benchmarking
application of mathematics and computation based methods for benchmarking a group of entities
aims at developing objective and automated methods of benchmarking.
Overwhelming majority of literature focuses on
developing new benchmarking methodologies
An important aspect forgotten:
post-analysis of the benchmarking results

13. Motivation Data Mining Overwhelming majority of literature focuses on
growing field of computer science
aims at discovering the hidden patterns and coming up with actionable insights.
Overwhelming majority of literature focuses on
developing more efficient and effective computational algorithms.
Important aspects not drawing deserved attention:
the quest for practical actionable knowledge
data mining can be used for post-analysis of results of other methodologies & algorithms

14. This Seminar Goals Case study applications
SmartDEA Solver framework for integrating analytical benchmarking with data mining
How DEA results should be structured
Meaningful interpretation of DEA results
Case study applications
Automotive
Wind energy
Apparel retail

15. Research Questions
How can Data Envelopment Analysis (DEA) results be structured such that they can be analyzed using readily available data mining techniques and software tools? (SmartDEA)
How can DEA & information visualization be used together? (Case Study 1)
Which visualization techniques are appropriate for analyzing DEA results? (Case Study 2)
How can DEA and data mining be integrated with the results of other data mining techniques, specifically association mining results? (Case Study 3)

16. Presentation Contents
Background on Data Envelopment Analysis (DEA)
SmartDEA framework
Case Studies
Automotive
Wind Energy
Apparel Retail

17. Background

18. Sample DEA Analysis

19. Data Envelopment Analysis (DEA)
Entities = DMUs (n DMUs)
Comparison of DMUs
Inputs and outputs (m inputs, s outputs)
Results
Efficiency score between 0 and 1
Reference sets
Projections

20. Basic DEA Models
Maximize the ratio : 𝑉𝑖𝑟𝑡𝑢𝑎𝑙 𝑖𝑛𝑝𝑢𝑡 𝑉𝑖𝑟𝑡𝑢𝑎𝑙 𝑜𝑢𝑡𝑝𝑢𝑡 for each DMU0

21. Basic DEA Models
CRR-Input model
CRR-Output model

22. Basic DEA Models
BCC-Input model
BCC-Output model

23. Basic DEA Models

24. Alp Eren Akcay, Gürdal Ertek, Gulcin Buyukozkan
Analyzing the solutions of DEA through information visualization and data mining techniques: SmartDEA Framework
Alp Eren Akcay, Gürdal Ertek, Gulcin Buyukozkan
Gurdal Ertek

25. Research Questions
How can Data Envelopment Analysis (DEA) results be structured such that they can be analyzed using readily available data mining techniques and software tools? (SmartDEA)
How can DEA & information visualization be used together?
Which visualization techniques are appropriate for analyzing DEA results?
How can DEA and data mining be integrated with the results of other data mining techniques, specifically association mining results?

26. Goal
To build a framework for making analytical benchmarking and performance evaluations
To design and develop a convenient DEA software, SmartDEA

27. Contribution To develop a general framework
To help DEA analysts to generate important and interesting insights systematically
To integrate the results for information visualization techniques

28. Framework
Integration of DEA results with data mining and information visualization

29. Proposed framework
integrates data mining and information visualization with DEA,
generates clean data for mining (data auditing at the DEA modeling stage),
allows the incorporation of “other data” into the process,
can accommodate multiple DEA models within same analysis.

30. Notation

31. Notation

32. Notation

33. Notation

34. Notation

35. Notation

36. Notation

37. Notation

38. Notation

39. SmartDEA: the developed software

40. Modeling Process C# language Results in file format of MS Excel
Imported data requires a certain format

41. Modeling Process 1- Importing Excel File:
Data requires a certain format

42. Modeling Process
2- Selecting the spreadsheet:

43. Modeling Process
3- Constructing the model:

44. Modeling Process
4-Selecting the DEA Model:

45. Modeling Process
5- Solving and generating the solution file:

47. Dr. Gürdal Ertek, Tuna Çaprak
Case Study 1: Integrating DEA with Information Visualization for Benchmarking Dealers in the Automotive Industry
Dr. Gürdal Ertek, Tuna Çaprak

48. Research Questions
How can Data Envelopment Analysis (DEA) results be structured such that they can be analyzed using readily available data mining techniques and software tools?
How can DEA & information visualization be used together? (Case Study 1)
Which visualization techniques are appropriate for analyzing DEA results?
How can DEA and data mining be integrated with the results of other data mining techniques, specifically association mining results?

49. A New Approach for Benchmarking and Managing TOFAŞ Dealers
Tuna Çaprak
Leaders for Industry Program ’07-’08, Sabancı University
Gürdal Ertek, Ph.D.
Faculty of Engineering and Natural Sciences, Sabancı University

50. A New Approach for Benchmarking and Managing TOFAŞ Dealers

51. Data Envelopment Analysis (DEA)
Benchmark Independent Decision Making Units (DMUs)
Consider Multidimensional Input / Output Relations
Express Efficiency with a Single Score Between 0 and 1

52. Reveal Hidden Structures Identify Patterns Derive Actionable Insights
Information Visualization (InfoViz)
Reveal Hidden Structures
Identify Patterns
Derive Actionable Insights

53. Develop Competitive Strategies
Information Visualization (InfoViz)
Reveal Hidden Structures
Identify Patterns
Derive Actionable Insights
Develop Competitive
Strategies

54. Model 1: Measuring “Efficiency”

55. Model 1: Measuring “Efficiency”
I N P U T S
O U T P U T
Dealer Expenses
DMU
Dealer
Spare Parts Area
Revenue (Total)
No of Employees

56. Model 2: Measuring “Efficiency for TOFAŞ”

57. Model 2: Measuring “Efficiency for TOFAŞ”
I N P U T S
O U T P U T
Dealer Expenses
DMU
Dealer
Spare Parts Area
Amount Purchased from TOFAŞ (YTL)
No of Employees

58. Other Data of Interest on DMUs
Share of TOFAŞ
IsRentEstimated
Cities
No of Services

59. ANALYSIS and DISCUSSIONS

60. Visualization of results
Miner 3D

61. Visualization of results
Omniscope

62. Incorporation of City Growths Technical Report and Paper
Future Work
Further Data Analysis
Incorporation of City Growths
Technical Report and Paper

63. Special Thanks to …
Prof. Muhittin Oral
Sinan Südütemiz
Hasan Erdoğan

64. Dr. Gürdal Ertek, Murat Mustafa Tunç Ece Kurtaraner, Doğancan Kebude
Case Study 2: Insights into the Efficiencies of On-Shore Wind Turbines: A Data-Centric Analysis
Dr. Gürdal Ertek, Murat Mustafa Tunç
Ece Kurtaraner, Doğancan Kebude

65. Research Questions
How can Data Envelopment Analysis (DEA) results be structured such that they can be analyzed using readily available data mining techniques and software tools?
How can DEA & information visualization be used together?
Which visualization techniques are appropriate for analyzing DEA results? (Case Study 2)
How can DEA and data mining be integrated with the results of other data mining techniques, specifically association mining results?

66. Outline Wind Turbines Our Study Methodology : Analysis and Results
Data Envelopment Analysis (DEA)
Visual Data Analysis
Hypothesis Testing
Analysis and Results
Insights
Brief outline of my presentattion

67. Wind Turbines
Mechatronic devices that convert wind energy into electrical energy via mechanical energy.
Features:
Diameter
Air dynamics
Tower height
Controlling devices
Location (On-shore / Off-shore)

68. Importance of Wind Turbines
Green Energy
Worldwide installed wind power capacity
In 1990: 2,160 MW
In 2011: 238,351 MW (Global Wind Energy Council)
16% of Europe’s electricity by (The European Wind Energy Association)
İmportance of green energies is known worldwide
İncreased around 120 times
Wind power can generate the 16% of europes electricity by 2020

69. Wind Energy in Turkey 40 GW wind energy potential in next 20 years
Image Source:

70. Wind Energy in Turkey
MİLRES: 500kW wind turbine to be designed and made in Turkey,
In 2013 output of 500 kW
In 2015 output of 2 MW
Largest budget civilian R&D project in the history of the Turkish Republic

71. Our Study
Technical data of wind turbines are collected and analysed by following methodologies:
Data Envelopment Analysis (DEA)
Visual Data Analysis
Hypothesis Testing
Aim:
Decision of the efficient wind turbines
Understanding of how to make an unefficient turbine efficient by referencing the efficient ones
Benchmarking of commercial wind turbines visually and statistically.
Bunun gramatiğini kontrol et

72. Literature First example of: Use of DEA and visualization together:
Benchmarking of commercial wind turbines
Visualisation as a directed graph of reference sets in DEA results
Use of DEA and visualization together:
Ertek et al. (2007) “Benchmarking the Turkish apparel retail industry”.
Ulus et al. (2006) “Financial benchmarking of transportation companies in the New York Stock Exchange (NYSE)”.

73. Methodologies Data Envelopment Analysis
Efficiency comparision of Decision Making Units (DMU) according to
Inputs (lower)
Outputs (higher)
For each DMU
Efficiency score (between 0 and 1)
Reference sets
Projections

74. Methodologies Visual Data Analysis
To distinguish different patterns in data and achieve new and useful insights. (Keim, 2002)
Orange Canvas (software)
Scatter plot
Miner 3d (software)
Surface plot

75. Top 10 companies in worldwide market share
Database
Top 10 companies in worldwide market share
Vestas (Denmark)
Sinovel (China)
Goldwind (China)
Gamesa (Spain)
Enercon (Germany)
GE (USA)
Suzlon (India)
Guodian (China)
Siemens (Germany)
Ming Yang (China)

76. DEA Model Model A : 74 on-shore wind turbine models
Model B : 32 on-shore wind turbine models (low-wind)
Inputs:
- Diameter (m)
- Nominal wind speed (m/s)
Outputs:
- Nominal Output (V)
Other features:
Cut-in wind speed (low/medium/high)
Company
We especially focused on on shore and low wind turbine models, because of milres project, that models are more suitable for turkey

77. DEA Model “BCC Output Oriented” Smart DEA Solver software
Developed in Sabancı University
Reads data from MS Excel and generate results
Visual analysis with Orange Canvas and Miner3D using efficiency scores

78. Analysis and Results

79. 1 - Efficiency vs Companies
Enercon and GE tend to have higher efficiency scores

80. 2 - Efficiency vs Nominal Output
A – efficiency is not significantly depend on nominal output, there are turbines that have efficiency score of 1 but with different nominal outputs
B – there is a cluster of turbines with 1500 kw output that have higher efficiency score

81. 3 - Efficiency vs Cut-in Wind Speed
Cut in wind speed : min. Wind speed that turbine can start to operate
Threre are more observations for cut in speed of 3, 3.5 and 4 m/s

82. 4 - Efficiency vs Diameter
A - the turbine with the min diametwer is efficient
B - and the turbine with the second min diameter has a high efficiency score
C – there is a cluster of turbines with diameter between 70 and 85 meters

83. 5 - Reference Analysis
Which efficient turbine models should inefficient ones take as references?
X axis: Efficient turbine model that should taken as reference
Y axis: DMU name
Size of circle: Weight of reference

84. A – this line shows the turbines with efficiency score of 1
B – some turbines like no 49 should primarly take as example only one model
C – some turbines like no 66 have balanced distribution of reference set

85. 6 - Reference sets for Model B with yEd software
This is for model b (low wind cut in wind speeds)
Dmu and 49 are the ones that take as a reference at most

86. 7 - Projection Analysis
At which percentage should the models change their inputs and outputs to become efficient?
X-axis : Percentage change
Y-axis : Efficiency
Colors: Inputs and outputs

87. A - Some of the models should both decrease inputs and increase outputs to become efficient
B - For most of the models it’s enough to increase outputs

88. 8 - Miner 3D Surface Plot Analysis
Shows the interaction between nominal wind speed, nominal output and diameter
This regions parameters result in low efficincy scores so should be avoided

89. 9 - Miner 3D Surface Plot Analysis
İnteraction between diameter cut in wind speed and nominal output
For high cut in wind speed, the region a and c should be avoided, diameter and nominal output should be aranged like in region b for higher efficiency scores

90. Insights Efficiency according to companies:
Enercon and GE are the most efficient companies
The efficiencies of turbines of Goldwind, Ming Yang, Mitsubishi and Siemens are under 60%
Efficiency according to nominal output:
Lower or higher values of nominal output is not effect efficiency
But, outputs around 1.5 MW have higher efficiencies
This insights can help manaers in investment decisions and design engineers

91. Insights Efficiency according to cut-in wind speed:
2 and 2.5 m/s have lower; 3, 3.5 and 4 m/s have higer number of models
3 m/s and over have higher efficiency scores compared to 2 and 2.5 m/s
Efficiency according to diameter:
Model with the smallest diameter is the most efficient turbine
Efficiency score of models with diameter between 70m and 85m are higher than expected

92. Insights Reference analysis: Projection analysis:
DMUs 15, 20, 27, 61, 81 are the ones that taken as a reference at most
Projection analysis:
Some of the models should both decrease inputs and increase outputs to become efficient
For most of the models it’s enough to increase outputs
Miner 3D surface plot analysis:
Input and outputs parameters of the models in light colored regions are ideal for higher efficiency

93. Hypothesis Testing Kruskal – Wallis Test confirmed that:
Efficiency scores and cut-in wind speed is significantly different depending on the companies.

94. References
Cooper, W. W., Seiford, L. M., Tone, K. (2006), Introduction to Data Envelopment Analysis and its Uses, Springer, New York.
Ertek, G., Can, M.A., Ulus, F. (2007) “Benchmarking the Turkish apparel retail industry through data envelopment analysis (DEA) and data visualization”. In: EUROMA th International Annual EurOMA Conference: Managing Operations in an Expanding, Ankara, Turkey
Keim, D. A. (2002), “Information visualization and data mining,” IEEE Transactions on Visualization and Computer Graphics, Vol.8, No.1, pp. 1-8.
Ulus, Firdevs and Köse, Özlem and Ertek, Gürdal and Şen, Simay (2006) “Financial benchmarking of transportation companies in the New York Stock Exchange (NYSE) through data envolopment analaysis (DEA) and Visulation”. In: 4th International Logistics and Supply Chain Congress, İzmir, Turkey, İzmir
Weill, L. (2004), “Measuring cost efficiency in European banking: a comparison of frontier techniques,” Journal of Productivity Analysis, Vol.21, No.2, pp

95. Q&A Dr. Gürdal Ertek (ertekg@sabanciuniv.edu)
Murat Mustafa Tunç
Ece Kurtaraner
Doğancan Kebude

96. IDETC/CIE 2012 Gurdal Ertek, Murat Mustafa Tunc ertekg@sabanciuniv.edu
Case Study 3: Re-Mining Association Mining Results Through Visualization, Data Envelopment Analysis, and Decision Trees
Gurdal Ertek, Murat Mustafa Tunc

97. Research Questions
How can Data Envelopment Analysis (DEA) results be structured such that they can be analyzed using readily available data mining techniques and software tools?
How can DEA & information visualization be used together?
Which visualization techniques are appropriate for analyzing DEA results?
How can DEA and data mining be integrated with the results of other data mining techniques, specifically association mining results? (Case Study 3)

98. Book Chapter Published in
‘‘Computational Intelligence Applications in Industrial Engineering’’
A book edited by Prof. Cengiz Kahraman
Published by Atlantis & Springer

99. Outline Introduction Literature Methodology Case Study Conclusion
Data Analysis
Data Visualization
Data Envelopment Analysis
Decision Trees
Classification
Conclusion

100. Introduction
How the results of association mining analysis further analyzed using
Data visualization
Data Envelopment Analysis (DEA)
Decision Trees
Visual Re-Mining of an item considering both
Positive assocations
Negative associations

101. Association Mining Inputs: Outputs: Primary metrics:
Transaction data that contains a subset of items
Outputs:
List of item-set that appear together frequently
Primary metrics:
Support is the percentage of transactions that the items appear in
Confidence is the conditional probability that item B appearing in transaction given that item A readily appears

102. Association Mining
A classical application is market basket analysis

103. Graph Visualization Refers to the drawing of graphs, that consists
Nodes
Arcs
Special algorithms
In order to obtain actionable insights

104. Re-mining
Mining of a newly formed data constructed upon the results of data mining process
The goal is
to obtain new insights that couldn’t have been discovered otherwise, and
to characterize, describe, and explain the results of the original data mining process

105. Data Envelopment Analysis
Benchmark a group of entities through efficient scores
Entities are called Decision Making Units (DMUs)
Efficiency score increases, if
DMU generates higher output using same input, or
DMU uses less input for the same output

106. Graph Metrics Degree shows the number of connections
Betweenness centrality represents total number of shortest paths
Closeness centrality shows the distance between the node and every other node
Eigenvector centrality shows the distance between the node and every other “special” node

107. Graph Metrics
Page rank is the value that increases if node is closely related with “special” nodes
Clustering coefficient represents the tendency of aggregation for several nodes

108. Decision Trees
Main goal: To identify the nodes that differs considerably from its root node
Each node is split (branced) according to a criterion
Our study uses ID3 algorithm
Branches are created in Orange software

109. Classification
Dataset is divided into two groups, namely learning dataset and test dataset
Classification algorithms are called learners
Naive Bayes
k-Nearest Neighbor (kNN)
C4.5
Support Vector Machines (SVM)
Decision Trees
The prediction success of each learner is measured through classification accuracy (CA)

110. Methodology Perform positive association mining
Find negatively association item pairs from 1.
Compute the percentage of positive associations
Construct two association graphs, (1) shows only positive assoc., (2) shows only negative
Compute graph metrics for each node

111. Methodology 6. Construct the dataset for re-mining
Apply grid layout for graphs, then visually analyze them.
Construct a DEA model, to combine the insights and to find the most important items
Construct a classification model and decision trees
Apply multiple learners and evaluate classification accuracy

112. Case Study Based on real company data in apperal retail industry
Merchandise group in men clothes line
2007 season

113. Case Study Company headquartered in Istanbul 300+ stores in Turkey
30+ stores in more than 10 countries

114. Case Study
As of Nov. 2010, the U.S. retail industry exceeded $377.5 billion

115. Data Analysis Step 1: Positive association mining
Min. support value : 100
Result: 3930 frequent item pairs involving 538 items
Step 2: Negative association mining
Result: 2433 item pairs involving 537 items
Step 3: Percentage of positive associations of each item

116. Data Analysis
Step 3: Percentage of positive associations of each item

117. Data Analysis
Step 4: Positive and negative association graphs

118. Data Analysis
Step 5: Graph metrics were computed using NodeXL add-in for MS Excel
Step 6: Dataset formed for re-mining
Each row is item involding positive association
Columns include
unique item number
support count (SupC)
StartWeek
EndWeek
LifeTime
MaxPrice
MinPrice
PriceDiff
MerchSubGroup
Category
PercOfPositiveAssoc
Graph Metrics

119. Data Visualization Step 7: Grid layout applied for visualization
Color denotes PercOfPositiveAssoc
Lighter items are mostly negative associated
Darker items are mostly positive associated

120. Data Visualization

121. Data Visualization Second graph:
IDETC/CIE 2012
Data Visualization
Second graph:
Node size represents end-of-season sales prices (MinPrice)
Larger nodes denote higher MinPrice (more typically high-priced items)
Smaller nodes denote lower MinPrice

122. Data Visualization

123. Data Visualization Third graph:
IDETC/CIE 2012
Data Visualization
Third graph:
Node shape represents category
We want to answer if the items have a particular category type
Upper left region
Darker nodes
Larger nodes

124. Data Visualization

125. Data Envelopment Analysis (DEA)
To analytically integrate the insights found in visualizations above
Input:
Uniform for each item
Output:
Support Count (SupC)
PercOfPositiveAssoc
MinPrice
Output oriented BCC model

126. Data Envelopment Analysis
INPUT
INPUT
OUTPUT
OUPUT
Item
Eff1*
Eff2**
Input_Auxiliary
Input_LifeTime
PercOfPositiveAssoc
SupC
MinPrice
059
Yes 1 16
91.67
4157
19.99
087
094
106 26 11 32
92.31
75.00
30.00
8947
4647
346933
14.90
41.57
9.25
169 No 7
4464
34.90
289 8
87.50
4317
23.06
412 13
88.89
2658
438 10
4999
513 4
80.00
5115
13.80

127. Conclusions Our methodology combines
Association mining
Graph theory
Classification
Data Envelopment Analysis
Re-mining
Positive associations are related to graph metric values and item’s attributes

128. References
A. Demiriz, G. Ertek, T. Atan and U. Kula, Re-mining item associations: Methodology and a case study in apparel retailing, Decision Support Systems, 52(1), pp (2011).
J.R. Quinlan,Induction of decision trees, Machine Learning, 1(1), pp (1986).
Orange.
E.Alpaydin, Introduction to Machine Learning,The MIT Press(2010).
A. Demiriz, G. Ertek, T. Atan and U. Kula, Re-mining item associations: Methodology and a case study in apparel retailing, Decision Support Systems, 52(1), pp (2011).
E.M.Bonsignore, C. Dunne, D.Rotman, M. Smith, T. Capone, D.L. Hansen andB. Shneiderman, First Steps to NetViz Nirvana: Evaluating Social Network Analysis with NodeXL,inInternational Symposium on Social Intelligence and Networking (2009).
R. Agrawal, T. Imielinski and A.N. Swami, Mining association rules between sets of items in large databases,in SIGMOD Conference,P. Buneman and S.Jajodia, (Eds) (1993).

129. References NodeXL. http://nodexl.codeplex.com/.
A.E. Akcay, G. Ertek and G. Buyukozkan, Analyzing the solutions of DEA through information visualization and data mining techniques: SmartDEA framework, Expert Systems with Applications (2012).
R.D. Banker, A. Charnesand W.W. Cooper, Some models for estimating technical and scale inefficiencies in data envelopment analysis,Management Science. 30(9), pp. 1078–1092. (1984).
G. Ertek and A. Demiriz, A framework for visualizing association mining results, Lecture Notes in Computer Science (LNCS), 4263, pp (2006)
G. Ertek, M. Kaya, C.Kefeli, O. Onurand K. Uzer, Scoring and Predicting Risk Preferences,in Behavior Computing: Modeling, Analysis, Mining and Decision, Cao, L., Yu, P. S. (Eds), Springer(2012).
C. Borgeltand R. Kruse, Graphical models: methods for data analysis and mining, Wiley (2002).
E.N. Cinicioglu, G. Ertek, D. Demirerand H.E. Yoruk,A framework for automated association mining over multiple databases, in Innovations in Intelligent Systems and Applications (INISTA), International Symposium, IEEE, (2011).

130. References
A. Savasere, E. Omiecinski and S. Navathe, Mining for strong negative associations in a large database of customer transactions, in Data Engineering, Proceedings., 14th International Conference, IEEE (1998).
P.N. Tan, V. Kumar and H.Kuno, in Western Users of SAS Software Conference (2001).
I. Herman, G. Melanconand M.S. Marshall, Graph visualization and navigation in information visualization: A survey, Visualization and Comp. Graphics, 6 (2000)
M. Van Kreveld and B. Speckmann, Graph Drawing,Lecture Notes in Computer Science (LNCS), 7034 (2012).
R. Spence, Information Visualization, ACM Press (2001).
H. Ltifi, B. Ayed, A.M. Alimiand S. Lepreux,Survey of information visualization techniques for exploitation in KDD, in Int. Conf. Comp. Sys.and App.(2009).
C. Chen, Information Visualization, Wiley Interdisciplinary Reviews: Computational Statistics, 2 (2010).
W.W. Cooper, L.M. Seiford and K. Tone, Introduction to Data Envelopment Analysis and Its Uses: With DEA Solver Software and References,Springer (2006).
S. Gattoufi, M. Oral and A. Reisman, Data envelopment analysis literature: A bibliography update ( ), Journal of Socio-Econ. Planning Sci., 38, pp (2004).

131. Gürdal Ertek ertekg@sabanciuniv.edu
Analytical Benchmarking Meets Data Mining: The SmartDEA Framework, SmartDEA Software, and Case Studies for Industry
Gürdal Ertek
Invited Seminar at A*Star SIMTECH, Singapore, August 2, 2013, Friday

132. Research Questions
How can Data Envelopment Analysis (DEA) results be structured such that they can be analyzed using readily available data mining techniques and software tools? (SmartDEA)
How can DEA & information visualization be used together? (Case Study 1, Automative)
Which visualization techniques are appropriate for analyzing DEA results? (Case Study 2, Wind)
How can DEA and data mining be integrated with the results of other data mining techniques, specifically association mining results? (Case Study 3, Apparel Retail)

133. Questions?

134. Thank you 感谢 Terima Kasih நன்றி Teşekkürler :-)