1. Oklahoma State University DelphiDr. Camille DeYongSlide 1 Expert Opinion Three circumstances –No historical data –Impact of external factors is more important than the factors that governed the previous development of technology –Ethical or moral considerations dominate

2. Oklahoma State University DelphiDr. Camille DeYongSlide 2 How to obtain expert opinion? Committees Advantages –Synergy –Total is greater than the sum of the parts

3. Oklahoma State University DelphiDr. Camille DeYongSlide 3 Disadvantages Misinformation Social pressure Group takes on a life of its own Repetition of arguments can influence opinion Vulnerability to dominant individuals Vested interests Entire group may share common biases

4. Oklahoma State University DelphiDr. Camille DeYongSlide 4 How to counteract this? Structured group processes –Delphi Process –Nominal Group Technique

5. Oklahoma State University DelphiDr. Camille DeYongSlide 5 Delphi Process Developed at the Rand Corporation as a means of extracting opinion from a group of experts. First dealt with a series of technological forecasts –Not just a TF method, however

6. Oklahoma State University DelphiDr. Camille DeYongSlide 6 Delphi Process Conducted in rounds Panel (participants) are not physically together Panel answers directed survey questions Moderator has responsibility for compiling and feeding back the input.

7. Oklahoma State University DelphiDr. Camille DeYongSlide 7 Three characteristics Anonymity Iteration with controlled feedback Statistical group response

8. Oklahoma State University DelphiDr. Camille DeYongSlide 8 Anonymity Avoids the possibility of associating responses with specific individuals Participants can change their minds without publicly admitting he/she has done so.

9. Oklahoma State University DelphiDr. Camille DeYongSlide 9 Iteration Interaction is conducted through responses to questionnaires Moderator extracts pertinent information

10. Oklahoma State University DelphiDr. Camille DeYongSlide 10 Statistical Group Response What kind of output do most groups produce? Delphi –Presents a statistical response –Opinions of entire group –Statistics that describe both the “center” and the “spread” Typically “quartiles”

11. Oklahoma State University DelphiDr. Camille DeYongSlide 11 Round 1 Typically is completely unstructured –Anticipate events or trends in the technology of interest –Advantages/disadvantages to this approach Questionnaires returned to moderator to compile –Similar items are combined, some dropped, events stated as clearly as possible –This is the questionnaire for the second round

12. Oklahoma State University DelphiDr. Camille DeYongSlide 12 Round 2 Consolidated list of events Asked to estimate time of occurrence Moderator prepares statistical summary –Median date –Upper and lower quartile dates

13. Oklahoma State University DelphiDr. Camille DeYongSlide 13 Round 3 Third questionnaire consists of the set of events and the statistical summary –Events, medians and quartiles Asked to prepare new forecasts for each event If their forecasts fall in the upper/lower quartiles, they are asked to support their position.

14. Oklahoma State University DelphiDr. Camille DeYongSlide 14 Round 3 (cont’d) Moderator prepares a statistical summary of forecast and summary of arguments Arguments/positions/objections are advanced just as they would be in a face-to- face meeting but, –interaction is anonymous and written.

15. Oklahoma State University DelphiDr. Camille DeYongSlide 15 Round 4 Questionnaire consists of –Lists of events –Medians/quartiles –Summary of arguments for changing the forecasts

16. Oklahoma State University DelphiDr. Camille DeYongSlide 16 Round 4 Panelists are asked to consider the reasons offered for changing the estimates and make new forecasts for each event. May be asked to justify position Moderator again computes medians and quartiles (and summarizes comments if necessary). Looking for “stability”

17. Oklahoma State University DelphiDr. Camille DeYongSlide 17 Outcome Events/trends Dates - usually use the median date in the fourth round Understanding of factors panelists believe are important and may affect the forecast.

18. Oklahoma State University DelphiDr. Camille DeYongSlide 18 Delphi vs. Committees Structured process for discussion and disagreement Panel members may or may not shift –Can do so without “losing face” Delphi generally results in a convergence of the panel estimates. –this is less frequently an outcome of an unstructured committee

19. Oklahoma State University DelphiDr. Camille DeYongSlide 19 Variations on Delphi Providing an initial list of events Beginning with a context Number of rounds Multiple dates Computerized Partial Anonymity

20. Oklahoma State University DelphiDr. Camille DeYongSlide 20 Important issues Group process Efficient Precision Selection of panel members Survey/questionnaire design

21. Oklahoma State University DelphiDr. Camille DeYongSlide 21 Success factors Selection of experts –Pay them? –Contact them personally –Explain the method Easy survey High response rate