Investigating the I MPACT OF LMI Research Team Réginald Savard, Sylvain Paquette, Céline Renald and Lynne Bezanson in collaboration with Bryan Hiebert Canadian Career Development Foundation (CCDF) Canadian Research Working Group in Evidence-Based Practice (CRWG) In partnership with New Brunswick Post-Secondary, Education and Labour

Outline Review of foundations for the study Principal findings Summary of results 2

Review of the Problem Most research on LMI focuses on usability of products: ▫ Readability, accuracy of information ▫ Ease of access, amount of use ▫ Most research is with students: very little with adults ▫ Little literature about the direct contribution of LMI ▫ Savard, R., Michaud, G., Bilodeau, C. et Arseneau, S. (2007). L’effet de l’information sur le marché du travail sur le processus décisionnel relatif au choix de carrière. Revue Canadienne de Counseling, 41(3), Several questions remain unanswered ▫ How do people use LMI? ▫ What (if any) assistance would be helpful? 3

Research Questions ▫ If client needs are assesed and clients are given LMI consistent with their needs: ▫ to what extent does assistance by a service provider enhance their effective use of LMI? ▫ and ▫ to what extent is independent self-help a sufficient process for clients to use LMI effectively? 4

Review of foundations for the study The study will take place within current service delivery practices in the employment centres ▫ so that processes can be incorporated into daily practice if research results prove positive. Career practitioners and clients will work in their customary settings LMI interventions will be isolated ▫ LMI is a distinct intervention, separate from employment counselling or worksearch workshops in which LMI may be a component ▫ There will be separate LMI interventions for career decision making and for job search 5

Procedures followed for the francophone sample Translation of LMI Booklets (client and resource centre) ▫ Career Decision Making (2) ▫ Work Search (2) Field testing ▫ Duration: 4 weeks Semi-structured interviews with clients ▫ 1 week and 4 months after entering the research Semi-structured interviews with research partners (career practitioners, managers) ▫ several weeks after the end of the research 6

Initial Contact: Employability Assessesment Interview Career Decision Making Work Search 7 Être apte au travail Choix de carrière Développement de compétences Recherche d’emploi Maintien en emploi Croissance professionnelle Dimensions de l’employabilité

Description of LMI Intervention 4 weeks of intervention Using the LMI booklets according to their identified need in ▫ Career Decision Making or ▫ Work Search Method of using LMI booklets: ▫ Independent ▫ Assisted ▫ Information and advice follow-up interviews (2) week one and and week 3 8 Career Decision Making Know yourself Know the labour market Arrive at a goal and develop an action plan Work Search Check for « fit » Get ready Search for work Secure employment

Client Sample 52 clients between18 and 56 years of age (average = 37 years) 25 men and 27 women 83 % had at least a secondary school diploma The majority worked in industry (eg.: seasonal workers, labourers) 81 % of clients were unemployed : 19 % were in full or part time employment 9 Two methods IndependentAssistedTotal Two kinds of interventions Career Decision Making WorkSearch11920 Total26 52

Career Practitioner Sample 15 career practitioners 3 men and12 women Number of clients per career practitioner Average = 3,47 (median = 3,87), range: between1 – 13. Between1 and 3 clients = 12 Between 4 and 6 clients = 0 More than 7 clients = 3 10

Methodology Post-pre self-assessment ▫ At the beginning of an intervention, people do not always know what they do not know….so there is a tendency to over or under-estimate ….. « Knowing what you know now (having completed the intervention), how would you rate your knowledge when you entered the research and how would you rate your knowledge now?» Dependent measures ▫ Knowledge of LMI ▫ Ability to use LMI ▫ Personal attributes (confidence and optimism) 11

Principal Findings 1.Clients on average demonstrated significant increases regardless of the method used (independent or assisted) 2.Clients in the assisted group demonstrated greater increases over time than the independent group. 3.Assisted clients in the career decision making group demonstrated greater increases than the other three sub-groups 12 1.Independent career decision making 2.Assisted career decision making 3.Independent work search 4.Assisted work search 4 sub-groups

First Major Finding Clients demonstrated on average significant increases regardless of method of intervention (assisted or independent) ▫ Between pre and post intervention: Overall ability to access and use LMI improved: ▫ Knowledge re how to use LMI ▫ Abilities to use LMI and to take action based on LMI ▫ Personal attributes: optimism, self-confidence and by inference, motivation 13

Sample of Results: Post-pre questionnnaire 0) Ne s’applique pas du tout 1)Ne s’applique pas du tout, mais presque 2)S’applique, mais tout juste bien 3)La réponse se situe entre tout juste bien et parfaitement bien 4)S’applique parfaitement 6/14 items NON Sur ces 6 items (Connaissances = 1, 2, 3, 5; Compétences = 4; Attributs personnels = 6)

Descriptive Results – POST-PRE Considering all 14 items in the questionnaire Before the interventionAfter the intervention 45 % of responses were not OK [0, 1]2 % of responses were not OK [0, 1] 55 % of responses were OK [2, 3, 4] 98 % of responses were OK [2, 3, 4] 8 % of responses were exceptional [4]52 % of responses were exceptional [4] 15

Descriptive Results– POST-PRE All the mean scores on the Before Intervention responses were in the Not OK range [0,1], with 1 exception (2,23). All the mean scores After the Intervention, without exception, were in the very OK or perfectly OK range [score average 3 or more]. The amount of change was similar across all dimensions of the survey same degree ▫ We found the same degree of change for knowledge, abilities and personal characteristics. 16

Second Major Finding Clients who were assisted by a career practitioner demonstrated greater change with time 17

Significant improvement in overall ability to use LMI between Before and After ▫ These results were consistent in all sub scales (knowledge, abilities, personal attributes) 18 Results Overall Score For the group as a whole: 14 items Post-Pre

Second Major Finding Assisted clients showed increased levels of change over time ▫ We can consider that having had two information and advice follow-up interviews, although brief, allowed assisted clients to state after the intervention, greater gains in knowledge and skill related to LMI as well as gains in confidence and optimism with respect to their career futures. 19

Information and Advice Interviews Two interviews of minutes each Three key steps in Information and Advice interviews: ▫ Re-establish the helping alliance and review the objective and action plan agreed to ▫ Give advice and information related to client use of LMI as appropriate for client need ▫ Come to agreement on next steps to move client action plan forward All Career practitioners reported following these 3 steps ▫ Importance was given to the helping alliance and to agreeing on the next steps to be followed Career practitioners used the second step to specifically respond to the information and action needs of the clients 20

Third Major Finding Assisted clients in Career Decision Making reported the highest gains among the four groups 1.Career Decision Making (independent) 2.Career Decision Making (assisted) 3.Work Search (independent) 4.Work Search (assisted) 21

Results showing both the two types of intervention and the two methods Final Question- naire Before Final Question- naire After Two Types of intervention Two methods n M (ET) M (ET) Time (p) Method (p) T x M (p) Career Decision Making Independent 1542,00 (17,78) 70,00 (13,79) 278,22 (< 0,01) 3,58 (0,07) 19,69 (< 0,01) Assisted 1724,65 (7,09) 72,94 (9,79) Total 3232,78 (15,69) 71,56 (11,73) Work SearchIndependent 1152,82 (24,31) 90,27 (12,25) 66,35 (< 0,01) 0,03 (0,86) 0,26 (0,62) Assisted 949,00 (25,64) 91,44 (13,16) Total 2051,10 (24,32) 90,80 (12,34) 22 F

Career Decision Making (assisted) (Suggested Interpretation) The unique position of clients who are trying to clarify a career direction may promote the asking of clarification questions which could increase the impact of an LMI intervention (Imel, Kerka et Wonacott, 2001) The advice given and the individual assistance may have resulted in an increase in the effectiveness of LMI for the individual (Brown et Ryan Krane, 2000) « The impact of the help of a career practitioner working with LMI and working directly with a client trying to make career planning decisions appears to be an important finding» (Savard, Michaud, Bilodeau et Arseneau, 2007, p. 163). 23

Other Results No significant differences for: ▫ Gender  Men and women responded similarly to both methods of working with LMI ▫ Age ▫ Employment history ▫ Unemployment history 24

Attribution of change To what extent would you say that the changes you reported before and after the intervention were the result of your participation in the research? Methods Mainly due to other factors Partly due to other factors Don’t know Partly due to this project Mainly due to this project Independent (n=26) Assisted (n= Total

Attribution of change 50 % of clients attributed their change to participating in the LMI research and not to other factors; 48 % attributed their change partially to their participation Overall there appear to be no significant differences in the results of the anglophone and francophone samples with the exception that assisted francophone clients reported increased gains over the anglophone clients although both groups reported more positive change in the assisted condition. 26

Client Engagement Client Engagement (client tracking sheets) Clients were very engaged in working with the LMI packages ▫ Clients accessed between 4 and 20 distinct LMI resources and made use of these between 7 and 148 times. ▫ On average clients accessed between 4 to 7 LMI resources; career decision making clients accessed on average 19 resources and work search clients accessed on average 18 resources. Most frequently used resources 27 Work Search LMI Booklets (90 %) Career Cruising (65 %) Career Decision Making Career Cruising (95 %) LMI Booklets (87 %) Travailler au Canada (87 %)

General Comments from clients Increase in confidence ▫ I doubted that I would benefit from the study based on past experience but I became more certain and confident in my own ability to find what I needed. ( 214) Better self awareness and knowledge of possible careers ▫ It was very useful to understand myself better and use this as a basis for searching for possibilities and making choices. (241) Overall a positive experience ▫ The study helped with knowing what direction to go in and I am sure that I have benefited a lot. I am more confident and pleased to have participated. (231) ▫ This allowed me to get more clear. I think I can continue to use this information as I face future changes. (217) 28

Client Criticisms The program needed to be longer given the number of resources available. (222) All the information I needed was available but I could not take it with me. It would have been better to have it all instead of having to go the the Resource Centre. […]. Working full time made it very hard […]. With a little more help it would have been much easier (242) 29

Client Criticisms There was too much information and that made making a career decision more difficult […] very hard to do this alone (209) The information in the booklets was too broad for my region […]. This could discourage certain people (252) I feel I am at the same point at the end of the study (237) 30

Follow up interviews (1 week and 4 months after intervention) Week 1: 39 clients (75% of original sample) 4 months: 27 clients (52% of original sample) ▫ LMI overall relevant ▫ Clients overall optimistic and confident ▫ Clients continued to use LMI but less often 31

Follow up Interviews Most useful aspects of LMI: ▫ Self-knowledge ▫ Job search techniques Overall weakness in service delivery: ▫ Action Planning 32

Follow-up interviews Most common concerns: ▫ « too much » information ▫ Lack of computer skills ▫ Lack of practitioner assistance especially:  Connecting self knowledge to world of work  Addressing employment barriers (confusion, disappointment) 33

For more information Réginald Savard, professeur et co-chercheur Paquette, professionnel de recherche Lynne Bezanson, CCDF, Site du GRDC < 34