1. Itamar Gati The Hebrew University Jerusalem
Evidence-Based Practice: Applying Decision-Theory to Facilitate Individual’s Career Choices
Itamar Gati
The Hebrew University Jerusalem

2. Choosing a Career as a Decision-Making Process: Unique Features
Amount of Information:
Often large N of alternatives
Large N of considerations and factors
Within-occupation variance
Practically unlimited
Quality of Information
Soft, subjective
Fuzzy
Inaccurate or biased

3. Unique Features of Career Decisions (continued)
Uncertainty
about the individual’s future preferences
about future career options
unpredictable changes and opportunities
the implementation of the choice
Non-cognitive Factors
emotional and personality-related factors
necessity for compromise
actual or perceived social barriers and biases

4. CDM Difficulties of 15,000 surfers on the Future Directions website (Gati & Meyers, 2003)
Are you experiencing difficulties in making your career decision?

5. Implications and Conclusion
Many factors contribute to the complexity and difficulties involved in the career decision-making process
Career counseling may be viewed as decision counseling, which aims at facilitating the clients' decision-making process, and promoting better career decisions
By adopting decision theory and adapting it to the unique features of career decisions, theoretical knowledge can be translated into practical interventions to facilitate individuals’ career choices

6. How can Theoretical Knowledge and Empirical Methods be used for Developing Counseling Instruments?
Today’s Presentation
The three bases of career counseling:
Locating the focuses of the client’s decision-making difficulties (CDDQ)
Guidance in the decision-making process
The three-stage model (PIC)
Identifying the client’s stage in the process
Characterizing the client’s decision-making style (DS)

7. Career Decision-Making Difficulties
The first step in helping individuals is to locate the focuses of the difficulties they face in making career decisions
Gati, Krausz, and Osipow (1996) proposed a taxonomy for describing the difficulties (see Figure 1), based on:
the stage in the decision-making process during which the difficulties typically arise
the similarity between the sources of the difficulties
the effects that the difficulties may have on the process and the relevant type of intervention

8. Figure 1: Locating Career Decision-making Difficulties based on the taxonomy of Gati, Krausz, & Osipow (1996)
Prior to Engaging in the Process
Lack of Readiness due to
Lack of motivation
Indeci-siveness
Dysfunc-tional beliefs
During the Process
Lack of Information about
Cdm process
Self
Occu-
pations
Ways of obtaining info.
Inconsistent Information due to
Unreliable Info.
Internal conflicts
Externalconflicts

9. The Career Decision-making Difficulties Questionnaire (CDDQ)
The Career Decision-making Difficulties Questionnaire (CDDQ) was developed to test this taxonomy and serve as a means for assessing individuals’ career decision-making difficulties
Cronbach Alpha internal consistency estimates: for the 3 major categories, .95 for the total CDDQ score

11. Empirical Structure of the Difficulties (N= 10,000; 2004)
Lack of motivations
Indecisiveness
Dysfunctional beliefs
Lack of info about self
Lack of info about process
LoI about occupations
LoI about addition sources of help
Unreliable Information
Internal conflicts
External conflicts

12. Computerized Assessment of Career Decision-Making Difficulties
The CDDQ was incorporated into a career-related self-help-oriented free of charge Internet site (
Research has shown that the Internet and the paper-and-pencil versions of the CDDQ are equivalent (Gati & Saka, 2001; Kleiman & Gati, 2004).
The CDDQ was found suitable for different countries and cultures and has been translated into 18 languages.

13. Interpreting the CDDQ results
Measuring career decision-making difficulties is not enough – interpretation is very important
Interpretation is part of face-to-face counseling and is crucial for Internet-based assessment of career decision-making difficulties, where no expert counselor is available
The proposed interpretation procedure is aimed at locating the individual’s salient difficulties and recommending ways to deal with them (with added reservations when needed)

14. The Four Stages of Interpretation
Ascertaining Credibility, using validity items and the time required to fill out the questionnaire
Estimating Differentiation based on the standard deviation of the 10 difficulty-scale scores
Locating the Salient, moderate, or negligible difficulties, based on the individual's absolute and relative scale scores
Determining the need to add reservations to the feedback provided (based on doubtful credibility, partial differentiation, or low informativeness)

15. The 4 Stages of Interpretation1
Evaluating
Credibility
Not Credible
Doubtful
Credible 2
Estimating Differentiation
Low
Questionable
High 3
Locate Salient
Difficulties
Aggregate
Reasons to Add Reservation (RAR)
Compute Informativeness
(B /W )
B/W < 1
RAR = 3
B/W > 1
RAR ≤ 2 4
Add Reservation
to Feedback
Receives Feedback
No Feedback

16. Interpreting the CDDQ results
The goal: empirically testing a four-stage model for interpreting the CDDQ profiles of individuals
The interpretation is based on the within-client relative salience of the difficulties as well as their absolute salience, augmented by quality-assurance measures
Career counselors' expert judgments were used to validate the proposed procedures of analyses

17. 5 Studies
Study 1: Ascertaining the Credibility of Responses to the CDDQ, based on validity items
Study 2: Estimating the Differentiation of Responses, based on the SDs of the 10 scale scores
Study 3: Determining the Relative Salience of Difficulties (salient, moderate, negligible)
Study 4: Determining the Need to Add Reservations to the Feedback

18. Studies 1-4
Career counselors' expert judgments were used in the four studies for validating the proposed procedures
Method
Participants: career counselors and graduate counseling students
Questionnaires: in studies 1,4 - all possible cases;
in studies 2,3 - responses of 16 actual clients
Results:
High similarity between experts’ and students’ judgments, as well as within-groups judgments
High similarity between the experts’ judgments and the proposed algorithm at each stage

19. Study 5 – Testing the Applicability of the Proposed Model
Method: Analyzing the CDDQ data of four groups (N = 6,192)
Hebrew paper-and-pencil version – 965 university students
Hebrew Internet version individuals surfing the Future Directions Internet site (
English paper-and-pencil version US College students
English Internet version individuals who filled out the CDDQ on the Internet ( )
Results: see Figures 3 & 4

20. Figure 3: The Distribution of the Three Levels of Difficulties (negligible, moderate, salient difficulty) in the Ten Difficulty Categories and in Four Groups (N = 6192; H-Hebrew, E-English, p-paper and pencil, I-Internet)
Difficulty category

21. Figure 4: Distribution of types of feedback in the four groups

22. Conclusions
The incorporation of a middle level of discrimination increases the usefulness of the feedback and decreases the chances and implications of potential errors
Adding reservations when appropriate is essential for providing meaningful feedback and decreasing the chances of misleading conclusions

23. General Feedback on the CDDQ

24. Detailed Feedback on the CDDQ

27. Among the salient difficulties is “lack of information about the career decision-making process” (4)
The Distribution of the Three Levels of Difficulties (negligible, moderate, salient difficulty) in the Ten Difficulty Categories and the Four Groups (N = 6192; H-Hebrew, E-English, p-paper and pencil, I-Internet)

28. Guidance in the decision-making process
The PIC model (Gati & Asher, 2001)
which separates the career decision- making process into 3 distinct stages:
- Prescreening
- In-depth exploration
- Choice

29. Prescreening
Goal: Locating a small set (about 7) of promising alternatives that deserve further, in-depth exploration
Method: Sequential Elimination
Locate and prioritize aspects or factors
Explicate within-aspect preferences
Eliminate incompatible alternatives
Check list of promising alternatives
Outcome: A list of verified promising alternatives worth further, in-depth exploration

30. Steps in Sequential Elimination
Locating and prioritizing aspects or factors
Explicate within-factor preferences in the most important factor not yet considered
Eliminate incompatible alternatives
yes
Too many promising alternatives? no
This is the recommended list of occupations
worth further, in-depth exploration

31. A Schematic Presentation of the Sequential Elimination Process (within aspects, across alternatives)
Potential Alternatives N
Aspects
a (most
important)
b (second in
importance) c . n
Promising Alternatives

32. In-depth exploration
Goal: Locating alternatives that are not only promising but indeed suitable for the individual.
Method: collecting additional information, focusing on one promising occupation at a time:
Is the occupation INDEED suitable for me?
verifying compatibility with one’s preferences in the most important aspects
considering compatibility within the less important aspects
Am I suitable for the occupation?
probability of actualization: previous studies, grades, achievements
fit with the core aspects of the occupation
Outcome: A few most suitable alternatives (about 3-4)

33. Choice
Goal: Choosing the most suitable alternative, and rank-ordering additional, second-best alternatives
Method:
comparing and evaluating the suitable alternatives
pinpointing the most suitable one
Am I likely to activate it?
if not - selecting second-best alternative(s)
if yes - Am I confident in my choice?
if not: Return to In-depth exploration stage
if yes: Done!
Outcome: The best alternative or a rank-order of the best alternatives

34. Still… But luckily, information and communication
Career decision-making requires collecting a vast amount of information
Complex information-processing is needed
But luckily,
information and communication
technologies are available
The use of a computer-assisted career guidance system based on a theoretical model can help overcome human cognitive limitations
There are several computer-assisted career guidance systems available, most of them on the Internet

35. However,
although Internet-based, career-related self-help sites are flourishing,
these sites, as well as “stand-alone” computer-assisted career-guidance systems, vary greatly in quality.
Hence,
it is very important to investigate the utility and validity of these self-help programs.

36. Stand-Alone, Internet-Based Career-Planning Systems
Possible Solutions
Desirable Features
CDDQ
Assessment of needs
Steps (PIC), factors to consider, dealing with compromises and uncertainty
Providing guidance concerning the process
potential alternatives, their characteristics, training
Providing relevant and accurate information

37. Stand-Alone Internet-Based Career-Planning Systems (continued)
Possible Solutions
Desirable Features
User’s input- continuous feedback, outcome – sensitivity analysis
Monitoring the dialogue
on the Internet or
elsewhere
Guiding the user toward additional sources of information
informative summary of the dialogue
Directing the user to face-to-face counseling when needed

38. MBCD Making Better Career Decisions
MBCD is an Internet-based career planning system that is a unique combination of
a career-information system
a decision-making support system
an expert system
Based on the rationale of the PIC model, MBCD is designed to help deliberating individuals make better career decisions

39. MBCD – Goals
Advancing the user’s career decision-making by locating a small set of promising occupational alternatives on which s/he may focus and collect more detailed information.
Increasing the user’s readiness and motivation to make a career decision.
Presenting a practical model of career decision-making that can be implemented in future career decisions as well as other decisions.

40. MBCD – System’s Features
Prescreening
Promising alternatives are located using the Sequential-Elimination model (Gati, 1986), which takes into consideration those career aspects that are most important to the counselee.
MBCD includes 28 career factors

42. MBCD’s Key Features (cont.)
Eliciting both facets of the individual’s preferences: (a) the optimal level
(b) additional levels that the user regards as acceptable (reflecting the user’s willingness to compromise)

45. MBCD’s Key Features (cont.)
Each occupation is characterized by a range of levels within each aspect, reflecting the within-occupation variance.
The system provides detailed feedback and recommendations according to the user’s input and its effect on the search results.
The dialogue is flexible and the users can change their responses at any point.

48. MBCD’s Key Features (cont.)
Promising alternatives are located by the Sequential-Elimination search
model (Gati, 1986).
But the user can also use a compensatory-model-based search.

49. Compensatory model-based search
Goal – locating the most compatible occupations
Rationale - advantages of occupations may compensate for their disadvantages
Steps of the compensatory search
Locate gaps between preferences and the characteristics of the occupation for each factor
Sum the gaps, weighted by importance of factors
Locate occupations with minimal sum of gaps

50. The Conjunction of the Two Lists
Sequential elimination-based list
Compensation-based list
Conjunction
list
Users are advised to focus on the occupations that were included in the recommended list of both search models in the in-depth exploration

52. MBCD’s Key Features (cont.)
Options to check the quality of the list of “promising occupations”, including:
“Almost compatible occupations” (i.e., sensitivity analysis)
“Why not”
“What if”
“Similar occupations”
“Compare Occupations”
During the dialogue users can ask which occupations have been eliminated from their list of promising alternatives due to only one small discrepancy with their reported preferences (this is the “almost compatible occupations” option).
The results of the sensitivity analysis are automatically included in the (personalized) dialogue-summary printout.

54. MBCD’s Features (cont.)
Initial in-depth explorations is offered by detailed occupational descriptions
During the dialogue users can ask which occupations have been eliminated from their list of promising alternatives due to only one small discrepancy with their reported preferences (this is the “almost compatible occupations” option).
The results of the sensitivity analysis are automatically included in the (personalized) dialogue-summary printout.

56. MBCD’s Features (cont.)
At the end of the dialogue
the user receives a printed summary to take along for further processing of the information. The printout also provides information for the counselor.
The user’s preferences are saved under a personalized code for future interactions.

57. Making Better Career Decisions
Does it really work?

58. END of PART 1

59. Making Better Career Decisions
Does it really work?

60. Prescreening Based on Elimination: Descriptive Validity (Gati & Tikotzki,1989)
The monitored dialogues of 384 career counselees with a computer-assisted career information system were analyzed.
Results: most users (96%) employed a non-compensatory strategy during all or at least a part of the dialogue: many options considered at a previous stage of the dialogue were not considered at the following stage, showing that individuals tend to use a prescreening strategy based on eliminating alternatives

61. Criteria for Testing the Benefits of Making Better Career Decisions
Examine users' perceptions of MBCD
Examine changes in user’s degree of decidedness
Examine perceived benefits
Locate factors that contribute to these variables

62. METHOD
Participants
247 males and 465 females who filled out both a pre-dialogue and a post-dialogue questionnaire
Mean age 22.8; mean years of education 12.6
4% high-school students
6% recent graduates from high school
58% recently completed their military service
9% considering an alternative to their current major
3% college graduates deliberating a job choice
8% considering a career transition
12% "other"

63. Decidedness Increased No change Decreased 355 (50%) 266 (37%) 91 (13%)
Mean Perceived Benefit (MPB) and Willingness to Recommend (WR) the Use of MBCD to a Friend (%) as a Function of the Difference in Decidedness after the Dialogue of MBCD (N=712)
Decidedness
Increased
No change
Decreased
Frequency
355
(50%)
266
(37%) 91
(13%)
MPB
3.12
2.57
2.52
WR%
93.5
74.8
72.5
Measure

64. Decidedness Before the Dialogue
Frequencies of Degree of Decidedness Before and after the Dialogue with MBCD
Decidedness
After the Dialogue
Decidedness Before the Dialogue 1 2 3 4 5
1- no direction 34 7 6
2 - only a general
direction 41 66 15 9
3 - Client is considering a
few specific alternatives 27 58 84 30
4 - would like to examine
additional alternatives 23 51 35 54
5 - would like to collect
information about a
specific occupation 20 21 28
6 - sure which
occupation to choose 16

65. Decidedness Before the Dialogue with MBCD
Willingness to Recommend (WR) the Use of MBCD to a friend as a Function of the Degree of Decidedness Before and After the Dialogue with MBCD (N=712)
Decidedness
After the Dialogue
with MBCD
Decidedness Before the Dialogue with MBCD 1 2 3 4 5
1- no direction 38 14 17 29 --
2 - only a general direction 85 73 67
100
3 - considering a few
specific alternatives 93 82 97
4 - client would like to examine
additional alternatives 92
5 - would like to collect information about a specific occupation 90 98 89
6 - Client is sure which
occupation to choose 81

66. MBCD’s Effect on Reducing Career Decision-Making Difficulties (d, Cohen, 1992)
Scale
.31
.13
.29
.16
Lack of Readiness
Motivation
General indecisiveness
Dysfunctional Beliefs
.72
.48
.45
.78
.20
Lack of Information About
The Process
The Self
Occupational Alternatives
Additional Sources
.11
.18
.01
-.13
Inconsistent Information
Unreliable Information
Internal Conflicts
External Conflicts
.65
Total CDDQ

67. MBCD’s Effect (d, Cohen, 1992) on Reducing Career Decision-Making Difficulties (Gati, Saka, & Krausz, 2003)

69. Predictive Validity of MBCD
Design: Comparing the Occupational Choice Satisfaction (OCS) of two groups:
those whose present occupation was included in MBCD’s recommended list
those whose present occupation was not included in MBCD’s recommended list

70. Method
Participants
The original sample included 123 clients who used MBCD in 1997, as part of their counseling at the Hadassah Career-Counseling Institute
Out of the 73 that were located after six+ years, 70 agreed to participate in the follow-up: 44 women (64%) and 26 men (36%), aged 23 to 51 (mean = 28.4, SD = 5.03)

71. Method
Instruments
MBCD
Questionnaire: clients were asked to report their field of studies, their satisfaction with their present occupational choice (scale of 1 – 9): “low” (1-4), “moderate” (5-7), “high” (8-9)
Procedure
the located clients were interviewed by phone, six+ years after visiting the career-counseling center

72. Results Frequencies of Occupational Choice Satisfaction by Acceptance and Rejection of MBCD's Recommendations, Based on Sequential Elimination

73. Frequencies of Occupational Choice Satisfaction by the Search-Model Whose Recommendations Were Accepted

74. Conclusions
Accepting the recommendations of the sequential-elimination-based search of MBCD produces the best outcomes (i.e., highest levels of satisfactions with the occupation)
The data does not support the effectiveness of the compensatory-based search
The data does not support any advantage of using the conjunction list over using only the sequential-elimination-search list

75. Alternative Explanations
Differences in the lengths of the lists
No difference was found in the OCS between clients whose list included 15 or fewer occupations and clients whose list included more than 15 occupations.
Therefore, this explanation can be ruled out.

76. Alternative Explanations (cont.)
Clients who accepted MBCD’s recommendations are more compliant, and therefore more inclined to report a high level of satisfaction.
However, following the compensatory-model-based recommendations did not contribute to the OCS.
Therefore, this explanation can be ruled out too.

77. Gender Differences in Directly and Indirectly Elicited Career-Related Preferences Gadassi and Gati 2006
Method
Participants. 226 females (74.1%) and 79 males (25.9%) who entered the Future Directions Internet site
Age: 17-30, mean=22.84 (median = 22, SD = 3.34)
Years of education: mean=12.67 (median 12, SD = 1.48)

78. Instruments Future Directions (http://www.kivunim.com)
Making Better Career Decisions (MBCD,
The preference questionnaire: this questionnaire imitated the preference elicitation in MBCD. Participants were presented with 31 aspects, and were asked to rank-order them according to importance, and to report their preferences in all 31 aspects

79. Preliminary analysis
Lists of occupations. We used MBCD to generate three lists of occupations according to:
(1) sequential-elimination
(2) compensation, and, for 235 participants,
(3) the list based on the conjunction between the sequential elimination and the compensatory search lists.

80. Preliminary analysis
Lists of occupations. We used MBCD to generate three lists of occupations according to:
sequential-elimination
compensation and, for 235 participants,
the list based on the conjunction between the sequential elimination and the compensatory search lists

81. Preliminary analysis
Determining the degree of gender-ratings of occupations was based on the judgments of 10 undergraduate students.
1 – “most (that is, over 80%) of the individuals who work in this occupation are women”
5 – “most (that is, over 80%) of the individuals who work in this occupation are men – over 80%"
The inter-judge reliability was .96,
We computed the mean gender-ratings of the lists of occupations for each participants

82. Preliminary analysis
Lists of occupations. We used MBCD to generate three lists of occupations according to:
sequential-elimination
compensation and, for 235 participants,
the list based on the conjunction between the sequential elimination and the compensatory search lists

83. Gender Differences in Directly and Indirectly Elicited Preferred Occupations (Gadassi & Gati, 2007)

84. Summary of Major Findings
PIC is compatible with people’s intuitive ways of making decisions (Gati & Tikotzki, 1989)
Most users reported progress in the career decision-making process (Gati, Kleiman, Saka, & Zakai, 2003)
Satisfaction was also reported among those who did not progress in the process
Users are “goal-directed” – the closer they are to making a decision, the more satisfied they are with MBCD
The list of Recommended Occupations are not sex-type biased (Gadassi & Gati, 2006)

85. Identifying the Client’s Stage in the Process
It is possible to start the PIC process from “the middle” – according to the client’s needs
However, it is recommended to start the process from the beginning, in order to:
Strengthen confidence in the occupational alternatives considered by the client
Eliminate inadequate alternatives considered by the client
Offer additional alternatives that were not considered by the client so far
Teach decisions skills: aspect-based instead of occupation-based approach

86. The stage in the PIC model decision-process of pre-academic programs students, at the beginning and end of the program (N=386)
The stage in the decision-making process – beginning of programs
total 4 3
The stage in the dcm process – end of programs 13 1 2
1-before pre-screening 77 5 17
2-before in-depths exploration 93 7 29
3- before choice
203 60 50
4 – after choice
386 73 98
Total - over rows
(55%) 211 made progress in the process
(35%) 136 stayed in the same stage
(10%) 39 moved backwards

87. Tailoring the Intervention to the Client’s Decision-Making Style
There is an advantage in tailoring the counseling intervention to the client’s decision-making style
Previous research typically characterized individuals by the most dominant characteristic of their decision-making style (e.g., intuitive, dependent).
we suggest that a multidimensional analysis should be used to uncover a comprehensive decision-making style-profile of clients.
A theoretical framework based on ten dimensions related to the career decision-making process was developed for characterizing individuals' career-decision making styles

88. The Ten Dimensions
The degree of analytic vs. holistic information-processing
The level of effort invested in the process
The degree of comprehensiveness in gathering and integrating the information
The degree of consultation with others
The degree of realism (willingness to compromise)
Internal vs. external locus of control
The speed of making the final decision
The degree of procrastination
The degree of dependence on others
The degree of acceptance to others’ wills

89. Testing the Proposed Model
To empirically test the proposed taxonomy we developed the career Decision-making Style Questionnaire (DSQ), in which each of the proposed dimensions was represented by a few statements.
The questionnaire was uploaded to a career-related, self-help oriented Internet site ( )
A cluster analysis supported the proposed differentiation between all ten dimensions.

92. Locating Repeated Profiles of Decision-Making Styles
Based on a cluster analysis of the participants, we located homogeneous groups of participants with similar career decision-making style profiles
We found five groups of participants with similar decision-making styles
These results were discussed in terms of the hypothesized ten dimensions and the previously identified career decision-making styles

93. The Means of the Located Groups in Terms of the 10 Dimensions Red = Low; Green = High5 4 3 2 1
Dimension
2.4
2.3
4.4
3.4
4.5
Analytic
2.2
3.3
4.2
3.9
4.6
Effort
3.5
Comprehens.
3.1
Consulting
3.7
4.0
2.7
3.2
3.6
Realistic
2.6
1.9
4.1
2.9
Locus of
Speed
Procrastin
4.9
Dependence
2.0
Acceptance

94. General Average of the Located GroupsSd M
Group
0.43
2.91 4
0.50
3.17 5
0.48
3.82 2
0.36
3.87 1
0.23
4.03 3

95. To sum up, I presented and discussed:
The CDDQ for locating the focuses of the individual’s decision-making difficulties, and the design and testing of a systematic procedure for interpreting its results
A general framework for cdm – the PIC model
MBCD – a unique combination of career information, expert, and a decision-support system
DSQ – A taxonomy and a questionnaire for a multidimensional analysis of client’s decision-making styles

96. To sum up
Career choices are decision-making processes, therefore career counseling is also decision counseling
Decision theory can be translated into practical interventions aimed at facilitating individuals’ career decision-making
Many tools were transformed into user-friendly Internet-based systems, which can be incorporated into counseling interventions
The theory-based interventions can and should be empirically tested for theoretical validity as well as practical effectiveness

98. END
Sofsof

99. Figure 2: Receives Feedback Ascertaining Credibility
credible
doubtful
high
partial
Locate Salient
Difficulty Categories
Add Reservation
to Feedback
low
No Feedback
Compute Informativeness
(Bv/Wv)
Receives Feedback
B/W > 1
B/W < 1
Estimating Differentiation
Ascertaining
Credibility
non
Aggregate
Reasons to Add Reservation (RAR)
RAR ≤ 2
RAR = 3
Figure 2:

100. Results: Compared Means of the Femininity-Masculinity Score According to Type of List and Gender

101. The Empirical Structure of the 10 Dimensions