1. Evaluating Immigrant Employment Systems Change Initiatives Rich Janzen September 2006

2.  Systems change and program evaluation  Government accountability frameworks  Immigrant employment research  Educational workshops An independent, not-profit social research organization with 23 years experience, including:

3. What is Systems Change? A system is a collection of parts that interact with one another to function as a whole (Manni & Cavana, 2002) Systems change can involve (Foster-Fishman, Egeren & Yang, 2005):  shifting the systems parts and/or their sequence  shifting interactions among parts  altering the “whole”  shifting how the system provides feedback to itself

4. Shifting System Parts Among the parts of the Immigrant employment system, the following characteristics can be altered:  Policies/practices  Resources  Opportunities  Relationships  Power/decision-making  Values  Attitudes/skills Adapted from Foster-Fishman, Egeren & Yang, 2005

5. The Immigrant Employment System Local Components Adapted from: “Making a Change Together”, Janzen et. al., 2001; Capacity Canada “National Review”, Janzen et. al., 2006 Immigrants Employers Provincial Components National Components Occupational Regulatory Bodies Ministry of Citizenship and Immigration Ministry of Training, Colleges and Universities Credential Assessment Services Professional Associations Professional Associations Human Resources & Social Development Canada Citizenship & Immigration Canada Embassies/ Consulates Regulating Ministries Canadian Heritage Industry Canada WRIEN Influences WIEN Facilitates Federal Politicians Provincial Politicians Immigrant Professional Associations National NGO’s Business Associations Non- government funders Provincial NGO’s Credential Assessment Services Educational Institutions Business Community/ Associations Immigrant Communities/ Associations Community Organizations Municipal Governments Non- Government Funders

6. Two Realities  In democratic societies, bringing about positive systems change is a long term process that evolves over time  Bringing about effective systems change requires a multi-faceted approach

7. Four Stages of Systems Change Stage 1 Steady State Stage 2 Build-up of Stress in the System Stage 3 Seen as a General Problem Stage 4 Resolution The system is OK Not all agree the system is OK, but most still do Most think the system is not OK The system starts changing Business as usual Normal channels fail Conditions ripen Trigger event and take off! Perception of failure Majority of public opinion Achieving alternatives Moving on The problem exists but it is not on the social and political agenda. Public not aware of the problem. People begin to point out that there is a problem. Opposition directed to existing decision- makers. Growing conditions for change. Some people become increasingly frustrated with the problem and the inability of power- holders to make the needed changes. A trigger event puts spotlight on a problem that violates widely-held values, sparking public attention and upset. A crisis atmosphere follows with many direct action campaigns. Many people working for change become disillusioned with lack of real change. Unrealistic hope of quick success is unmet. Some “burn out,” some turn more militant, acting on anger and despair (threatening to alienate movement). Movement transforms from protest in crisis to long-term struggle/nego- tiations with power-holders. Focus on winning public support and proposing alternatives. Movement’s position increasingly adopted as mainstream. A long process of proposing alternatives. Shift from “opposing” to “suggesting.” More costly for power-holders to continue old policies than to adopt new ones. Broad- based coalitions of support formed. Movement needs to protect and extend successes that were achieved. Long-term goal is to achieve a paradigm shift. People mostly work within the “new system” to push change forward. ▲▲ ▲ Initial Trigger More Trigger Events Event Adapted from Moyer, 1990 as found Janzen et. al., 2001 www.crehs.on.ca/downloads/handbook.pd f

8. Four Systems Change Activity Categories A multi-faceted response: 1. Public education 2. Policy influence 3. Planning and collaboration 4. Action research Source: Janzen et. al., 2006; in press

9. Individual Local Services Policy System Components Activity Categories Society Public Education Political Advocacy Community Planning Example of the Inter-Relatedness of Systems Change Activities Action Research Source: Adapted from Janzen et. al., in press

10. Systems Change Evaluation  Captures, rationalizes and assesses the evolving nature of system change interventions  Teases out direct outcomes that can be attributed specifically to the invention with indirect outcomes (the intervention only partially responsible for observed change)  Clarifies how changes in relationships among system players relates to observed outcomes  Assesses how the synergies of collaboration has led to changes not otherwise possible

11. For more details contact…. Rich Janzen, Research Director Centre for Research & Education in Human Services 73 King Street West, Suite 300 Kitchener, ON, Canada N2G 1A7 Phone: (519) 741-1318 Fax: (519) 741-8262 E-mail: rich@crehs.on.ca Website: www.crehs.on.ca