INTRODUCTION TO POLICY ANALYSIS Text:

INTRODUCTION TO POLICY ANALYSIS Text:
Patton and Sawicki, Basic Methods of Policy Analysis and Planning, 2nd edition.

THE NATURE OF PUBLIC PROBLEMS
In the public sector, problems: are fuzzy and ill-defined; have political as well as purely technical aspects; often lack a good cause-effect knowledge base;

may be solved only by producing new problems;
often involve tradeoffs between cost and effectiveness; may be hard to measure adequacy of results; may be hard to measure fairness of results.

CLASSICAL RATIONAL PROBLEM -SOLVING MODEL
In theory, problems can be approached using a rational, comprehensive problem solving model. The demands of this model are: 1) Define the problem 2) Determine important social values 3) Identify all alternatives 4) Assess all alternatives 5) Select optimal alternative 6) implement optimal alternative

LIMITATIONS IN THE PUBLIC SECTOR
Theoretical Model 1) Define the problem 2) Determine important social values 3) Identify all alternatives 4) Assess all alternatives on all values 5) Select optimal alternative 6) implement optimal alternative Public Sector Limitation 1) Problems are interlined 2) No agreement on social values 3) Limited time, knowledge 4) Limited resources, lack of predictability 5) Pressure to select the first good solution 6) Short time horizon to produce results

-Traditional research is concerned with broad, theoretical, complex questions.
It uses explicit scientific steps and invariant procedures. Policy analysis, on the other hand, is practical, situational and flexible. - It addresses local problems and focuses on making decisions. - It is more craft or art than science.

Traditional Research Seeks "truth" Explicit steps and procedures Addresses broad questions Focus on complexity Science Policy Analysis Is practical Flexible, situational Addresses local problems Focus on decision-making Craft

CASE STUDIES IN POLICY ANALYSIS
- Problems in the public sector are multi-faceted and difficult to pin down. - As if that was not bad enough, the knowledge domain of public policy is ill-structured. - This means that there is no "one best way" to solve all problems. - Giving policy analysis only one methodology is like giving a home owner only a hammer to solve all household problems.

- A new approach is needed to learning in this area.
This approach is offered by case studies. - Case studies link problems to a reality; - They offer the opportunity for the application of policy analysis techniques in a concrete context.

The way information is remembered and use is linked to the way it is learned.
- Case studies provide cues to the types of techniques that are needed to approach a solution to the problem. - These cues help policy analysts learn multiple approaches to learning and to problem solving.

Use of case studies will help to:
1) Recognize situations where analysis is appropriate and productive; 2) Become competent in the application of different approaches and methods; 3) Learn how to communicate the results of policy analysis.

TIPS FOR PRACTICAL POLICY ANALYSIS
1) Quickly identify the central decision criterion of the problem (What is the most important factor in buying a car? Taking a new job?) 2) Identify what types of public sector actions can be taken (Taxing, spending, sanctions, incentives, moral suasion, education?) 3) Avoid the "one best way" approach (Have many tools in the tool box, not just one)

4) Learn how to deal with uncertainty (Admit it, estimate its possible effects)
5) Say it with numbers (Charts, graphs, tables, maps, etc.) 6) Make the analysis simple and transparent (Provide details in a technical appendix) 7) Check and re-check the facts (Use multiple sources of facts, triangulation) 8) Learn to anticipate the objections of opponents (Improves the ultimate product)

9) Give analysis, not decisions (Distinguish between analysis and advocacy)
10) Push the boundaries of the envelope (Expand the problem definition; introduce novel solutions) 11) Policy analysis is never 100% complete, rational, and correct (How much time, money, and personnel is available to do the job?)

2. ALTERNATIVE POLICY ANALYSIS MODELS
1) Quade a. Policy formulation b. Search for alternatives c. Forecast the future d. Model the impacts of the alternative e. Evaluate, compare, and rank the alternatives

2) MacRae and Wilde a. Define the problem b. Determine criteria c. Generate alternatives d. Choose course of action e. Evaluate policy after implementation

3) Stokey and Zeckhauser
a. Determine the underlying problem b. Determine the objectives c. Generate alternatives d. Predict consequences of each alternative e. Determine criteria for measuring achievements f. Choose course of action

4) Urban Institute a. Define the problem b. Identify objectives c. Select criteria d. Specify the client e. Calculate the cost of each alternative f. Assess the effectiveness of each alternative g. Present the findings

5) Weiner and Vining a. Problem analysis a.1. Understand the problem a.2. Choose goals and constraints a.3. Choose method of solution b. Solution analysis b.1. Choose evaluation criteria b.2. Specify alternatives b.3. Assess alternatives b.4. Recommend solution

6) Hill a. Define problem b. Identify alternatives c. Quantify alternatives d. Apply decision aids e. Choose alternative f. Implement solution

7) Patton and Sawicki a. Verify, define and detail the problem b. Establish evaluation criteria c. Identify alternative policies d. Assess alternative policies e. Display and distinguish among alternatives f. Implement, monitor, and evaluate the policy

SIX STEP POLICY ANALYSIS
1) Verify, define and detail the problem 2) Establish evaluation criteria 3) Identify alternative policies 4) Assess alternative policies 5) Display and distinguish among alternatives 6) Implement, monitor, and evaluate the policy

VERIFY, DEFINE AND DETAIL THE PROBLEM
State the problem meaningfully: Determine the magnitude and extent of the problem Continually redefine the problem in light of what is possible Eliminate irrelevant material Question the accepted thinking about the problem Question initial formulations of the problem Say it with data Locate similar policy analyses Locate relevant sources of data

Eliminate ambiguity Clarify objectives Resolve conflicting goals Focus on the central, critical factors Is it important? Is it unusual? Can it be solved? Identify who is concerned, and why? What power do concerned parties have? Make a quick estimate of resources required to deal with the problem

2) ESTABLISH EVALUATION CRITERIA
What are the important policy goals, and how will they be measured? Identify criteria central to the problem and relevant to the stakeholders Clarify goals, values and objectives Identify desirable and undesirable outcomes Is there a rank order of importance among the criteria? What will be the rules for comparing alternatives?

Administrative Ease Costs and benefits Effectiveness Equity Legality Political acceptability

3) IDENTIFY ALTERNATIVE POLICIES
Consider a wide range of options Consider the status quo, or no-action alternative Consult with experts Brainstorming, Delphi, Scenario writing Redefine the problem if necessary

4) ASSESS ALTERNATIVE POLICIES
Select appropriate methods and apply them correctly Estimate expected outcomes, effects, and impacts of each policy alternative Do the predicted outcomes meet the desired goals? Can some alternatives be quickly discarded? Continue in-depth analysis of alternatives that make the first cut

5) DISPLAY AND DISTINGUISH AMONG ALTERNATIVES
Choose a format for display Show strengths and weaknesses of each alternative Describe the best and worst case scenario for each alternative Use matrices, reports, lists, charts, scenarios, arguments

6) IMPLEMENT, MONITOR, AND EVALUATE THE POLICY
Draw up a plan for implementation Design monitoring system Suggest design for policy evaluation Was the policy properly implemented? Did the policy have the intended effect(s)?

ROLE OF THE POLICY ANALYST
Policy analysis is a systematic evaluation of the technical and political implications of alternatives proposed to solve public problems. Policy analysis refers to both the process of assessing policies or programs, and the product of that analysis. A policy analyst: uses qualitative and quantitative data; uses a variety of approaches to the problem; applies appropriate methods correctly.

Who does policy analysis?
Is public policy analysis a calling? A vocation? A service? A guild? A cult? The role of the policy analyst is to: Produce arguments for debates about public policy Produce evidence for decisions about public policy Act as internal organizational consultants Act as external policy consultants Handle both technical and people aspects of policy analysis

All policy represents the distribution of power and resources
All policy represents the distribution of power and resources. These policies are an expression of values. Values and beliefs are often used as short-cuts to decision-making. What code of ethics should the policy analyst adopt? What about the professional values of obligation, responsibility, discretion, and citizenship? What about published professional codes of ethics, such as ASPA, ICMA, AICP, NASW, NSPE, etc.?

The policy analyst has responsibilities, to the client, the customer, the self, the profession, the public interest, fairness, equity, law, justice, efficiency, effectiveness, and the practice itself. Who is to define what is good? Whose values or goals should be pursued? What is the right thing to do? Who or what is ultimately to be served? Should the analyst try first and foremost to do good, or to do no harm? Should the analyst give neutral advice, or normative advocacy? Should the analyst be supportive or adversarial?

Bias is inevitable in policy analysis
Bias is inevitable in policy analysis. To mitigate the effects of bias, the analyst can: identify all underlying assumptions keep accurate records use multiple sources of information use replicable methods and models identify the client's goals and values identify the formal and informal actors and institutions address relevant professional and ethical considerations

3. CROSS-CUTTING METHODS
SELECTING TECHNIQUES Selecting the appropriate techniques to use in policy analysis depends on a variety of factors: what the client wants to know the time available knowledge of the decision criteria complexity of the issue available data

Some techniques commonly used in various stages of policy analysis include:
1. Verifying, Defining, and Detailing the Problem Back-of-the-envelope calculations Quick decision analysis Creation of valid operational definitions Political analysis Issue paper/first cut analysis

2. Establishing Evaluation Criteria
Technical feasibility studies Economic and financial feasibility studies Political viability studies Administrative operability studies

3. Identifying Alternatives
Researched analysis No-action analysis Quick surveys Literature reviews Comparison of real-world experiences Passive collection and classification

Development of typologies
Analogy, metaphor, and synectics Brainstorming Comparison with an ideal Feasible manipulations Modifying existing solutions

4. Assessing Alternative Policies
Extrapolation Theoretical forecasting Intuitive forecasting Discounting Cost/Benefit analysis Sensitivity analysis Allocation formulas Quick decision analysis Political feasibility analysis Implementation analysis Scenario writing

5. Displaying Alternatives and Distinguishing Among Them
Paired comparisons Satisficing Lexicographic ordering Non-dominated alternatives method Equivalent alternatives method Standard-alternative method Matrix display systems Scenario writing

6. Implementing, Monitoring, and Evaluating Policies
Before-and-after comparisons With-and-without comparisons Actual-versus-planned performance Experimental models Quasi-experimental models Cost-oriented approaches

7. Cross-Cutting Methods
Identifying and gathering data Library search methods Interviewing for policy data Quick surveys Basic data analysis Assessing information quality Communicating the analysis

CROSS-CUTTING METHODS
Cross-cutting methods are techniques of policy analysis that can be used at nearly any stage in the analysis. They are useful tools for the policy analyst to know how to use. They include: Identifying and gathering data Library search methods Interviewing for policy data Quick surveys Assessing information quality Basic data analysis Communicating the analysis

IDENTIFYING AND GATHERING DATA
Policy analysts need to know how to search for existing information, such as: academic journal articles archives census records hearings legislative history news media reports past policy analyses public agency reports public records

People are also good sources of information, including
advocacy groups experts issue networks personal contacts professional colleagues

Even personal observation can be a source of data
Even personal observation can be a source of data. Personal observation can furnish data on usage patterns, compliance patterns, insights into the problem, anecdotes, and innovative suggestions. However, observation is time consuming and may suffer from problems with accuracy, bias, limited samples, and difficult to quantify data. Observational methods include "sidewalk surveys," mechanical counting devices, measures of erosion, satellite images, etc.

Other sources of information include:
federal agencies libraries local agencies non-profit agencies private organizations research institutes state agencies think tanks universities

Policy analysts should seek information from multiple sources ("triangulation"), especially on controversial data. Problems with sources of data include: outdated statistics irrelevant data misleading data poor quality data biased data

Looking for documents that may be helpful in doing the policy analysis is important. But three questions that must be asked are: 1) do such documents exist? 2) can they be obtained in a reasonable time? 3) when is additional searching no longer worthwhile?

LIBRARY SEARCH METHODS
Libraries are excellent sources of policy-related information. To make the most of library resources, follow these strategies: 1) look up basic policy-related terms and definitions in encyclopedias, dictionaries, or a subject-related thesaurus; each policy issue area has its own terms and jargon 2) develop a list of search terms for searching computerized bibliographic data bases, electronic guides to library holdings, and Internet access;

3) identify key journals in the field and skim their table of contents for the past 1-2 years;
4) check guides to current periodicals, newspapers, news magazines, trade journals, and guides to the literature 5) check annual reviews in the policy subject area; conference proceedings on the subject; government hearings on the subject, etc.

The federal government offers a wide variety of sources:
Congressional Directory Government Yearbooks Guide to Federal Statistics International Statistics Population Reports Statistical Abstract of the U.S. U.S. Census U.S. Government Printing Office catalogues U.S. Government Manual

Many sources of legal information have bearing on policy issues:
Adjudication and case law Agency regulations Code of Federal Regulations Federal and State statutes Federal Register Legal Periodicals Municipal ordinances Supreme Court decisions

INTERVIEWING FOR POLICY DATA
Interviewing is typically conducted with either mass, elite, intensive, or focus group methodologies. Interviewing is typically used: to gather historical background, context, and evolution of the policy to gather basic facts about the problem to assess political attitudes and resources of major players to gather ideas about the future, trends, and forecasts to generate additional contacts and materials (snowball technique)

Elite (specialist) interviewing is most typically used when:
it is a short-term policy project it is on a new topic there is a lack of existing literature informants are reluctant to put information into writing no quantitative data are available it is not feasible to use hired interviewers

To set up interviews, the policy analyst usually:
arranges appointments in advance makes formal or informal requests (letterhead, telephone) sends a reminder letter and follows up with a phone call gives the name of a mutual friend or influential person as a reference collects background information prior to the interview will conduct a telephone interview if a face-to-face interview is not possible

When conducting the actual interview, it is usually accepted behavior to:
ask before using any recording device promise anonymity and/or confidentiality of information take notes during the interview keep to the allotted time thank the person for the interview send a follow-up letter

QUICK SURVEYS Surveys can be conducted by mail, in person, or by telephone. Survey methodology is described in many standard research texts. Cross-sectional interviews are conducted at one point in time across a wide sample of the population. Longitudinal interviews are conducted repeatedly over many time intervals (months, years, decades) with the same individuals. A comparison of the advantages and disadvantages of the most typical surveys is displayed below.

May not be representative Least staff time required
Type of Survey Response Rate Sample Concerns Staffing Other concerns Mail 15% May not be representative Least staff time required Response rate improves with gifts Telephone 50% Limited to those with telephones Moderate staff time required Short and simple questions In-Person 75% May be needed for less-educated Most intensive; most supervisors Can cover complex issues

ASSESSING INFORMATION QUALITY
When collecting information and data for policy analysis, the analyst must assess the quality of the information and data collected. Document Analysis: When was the document generated? What was the original purpose of the document? Is there an obvious bias in the document? What is the pattern of word usage? Does the document omit important information? Are there errors in the document?

Assessing Interviews:
Was the information plausible? Was the information consistent? Does the information diverge from accepted facts? Did the respondent report direct experience? Did the respondent have ulterior motives? Did the respondent operate under some constraints? Was the respondent candid? Did the respondent acknowledge areas of ignorance? Was the respondent self-critical?

Data quality: Are multiple sources of information consistent? Were data collected independently, from separate sources? Is the data original or re-organized? Do the data pertain to a particular geographic locale? Were the data collection methods systematic? For what purpose were the data originally collected? How old are the data? Were they affected by timing? Was there bias or special motivation in the collection of the data?

BASIC DATA ANALYSIS Data are not generally useful in their raw form. Instead, they must be analyzed. Data are most often analyzed using descriptive and/or inferential statistics. Descriptive statistics search for patterns in the data and look for relationships to gain insight into the problem. Inferential statistics attempt to estimate a characteristic of a population from data gathered from a sample.

Descriptive univariate statistics look for patterns in the data
Descriptive univariate statistics look for patterns in the data. They are best presented in graphical form, using frequency distributions, cumulative distributions, bar charts, histograms, pie charts, and frequency polygons. Statistics include the mean, median, and mode, as well as the range, standard deviation, and variance.

Descriptive bivariate statistics look for relationships in the data
Descriptive bivariate statistics look for relationships in the data. They are best presented in tables, plots, scattergrams, and time series graphs. Measures of association include Lambda, Gamma, and Pearson's r.

Inferential statistics make probabilistic statements (or inferences) about a whole population based on the results obtained from a partial sample. Measures of statistical significance are used to estimate whether two groups differ from one another, or whether there is a chance that a relationship observed in a sample also exists in a population. These measures include Chi Square, Z-scores, t-tests, and F-tests.

COMMUNICATING THE ANALYSIS
Written Communication: Work from an outline--keep separate folders for each section of the analysis Work from goals and deadlines--generate a complete draft and then fill in the holes Get help--with editing of rough drafts; revise for clarity; incorporate new ideas Include a Table of Contents--sections include Executive Summary; Problem Definition; Decision Criteria; Alternatives; Comparison of Alternatives; Conclusions and Recommendations;

Use graphics--charts, graphs, flow charts, tables, maps, pictures, diagrams, drawings, etc.
Use geographic information systems (GIS)--to generate maps of data distributions Simplicity--use the active voice for verbs Accuracy--verify facts; triangulate; check all calculations Documentation--note all formulas used and assumptions made Fairness--use references and give credit to your sources of data Neatness--use good grammar, spelling, punctuation, syntax, etc.

Oral Presentations Know your audience Keep it short and simple Use visual aids and handouts Allow time for questions, comments and criticisms

4. VERIFYING, DEFINING, AND DETAILING THE PROBLEM
PROBLEM DEFINITION The first thing the policy analyst must do is to ask: 1) Does a problem exist? 2) Can anything be done about it? 3) Does the client have the power? If the answers are no, then there is no point in doing a policy analysis.

Pitfalls in public policy problem definition:
1) accepting the client's definition of the problem 2) looking only for the simple and obvious 3) thinking that any and all problems need a public solution 4) confusing the need for short- versus long-term solutions 5) confusing the values of individuals versus collectivities

Don't Need Public Policies
Do Need Public Policies Individual problems Social problems Widespread problems Serious problems Relative problems Absolute problems

DEVELOPING PROBLEM STATEMENTS
In developing problem statements: 1) think about the problem 2) delineate the boundaries of the problem 3) develop a fact base 4) list goals and objectives for policy solutions 5) identify the policy envelope (key players) 6) develop preliminary costs and benefits 7) review the problem statement

BACK-OF-THE-ENVELOPE CALCULATIONS
One of the first things a policy analyst will do is to try to get a handle on the possible dimensions of the problem and potential solutions. The analyst may ask, 1) How many people are we talking about? 2) What is the likely cost per unit of service? 3) How much of the target population can we serve? 4) How much do we have available to spend? 5) Will more staff be needed? 6) Will this impact the budget/tax rate? 7) What are the trends in this area? 8) What will happen if we do nothing?

For example, try to estimate these parameters if half the children in the state are not receiving the required immunizations before beginning school. Start with the number of children in the state up to age 5. Which immunizations are required? How much does each one cost? How many children could realistically be reached? How much do we have available to spend? Could we get more from the Federal government? Will more state staff be needed, or can this be handled by the private/non-profit sector? Will this impact the budget/tax rate? What are the trends in this area--is the problem increasing or decreasing over time? What will happen if we do nothing?

The information for doing back-of-the-envelope calculations can come from
1) reference works 2) experts 3) past studies or quick research 4) informed guesses, extrapolation, rules of thumb, estimation, parallel reasoning, triangulation, etc.

QUICK DECISION ANALYSIS
Quick decision analysis is a variation on the technique of making decision trees. Decision trees are ways of diagramming a problem, when the problem has more than one solution. It is a tool to help policy analysts see the logical alternatives to a problem.

POLITICAL ANALYSIS Policy analysts recognize that politics is important at all stages of the policy process, including policy analysis. There are a number of ways to communicate about potential political influences or factors that may impinge on the policy analysis. These techniques attempt to allow political factors to be treated like any other important considerations in policy analysis.

The analyst may draw up a list of issues involved in defining the problem, and identify a number of potential political actors who have taken positions on those issues. A table can display the likely support or opposition of each group to each issue. For example, what are the issues involved in raising the age at which teens can get a driver's licence to 18? Which groups are likely to support (+) or oppose (-) problem definitions that focus on these issues?

Groups Lower teen auto accident, death & injury rates Hardship for teens who work or commute to school Lower insurance rates for family cars M.A.D.D. + ? Parents/Voters - Insurance Lobby

FIRST CUT POLICY ANALYSIS
An issue paper is a study that is conducted in preparation of making a decision on whether or not to do a policy analysis. It describes the problem, the attendant issues, the political groups involved, and concludes whether or not a policy analysis will be feasible.

A first cut policy analysis concentrates on identifying preliminary recommendations. It is a mini-policy analysis, conducted in a short period of time, using simple techniques. It forms the basis for a much more in-depth, complex, and thorough full-fledged policy analysis.

Researched analysis refers to a more traditional research project, perhaps conducting a pilot study of several policy alternatives to generate concrete data on which to base recommendations. However, policy analysts rarely have the luxury of the time and resources needed, nor do they often work for someone who is far enough removed from the problem to resist pressures for a quick solution.

5. ESTABLISHING ANALYSIS CRITERIA
WHAT ARE CRITERIA Every time a policy problem is identified, some statement of goals is adopted. The goals are what the adopted policy alternative should accomplish. Goals are broad, formal, long-term problem-solving achievements that are desired. An example might be to make sure that all rivers are safe, clean, and usable.

Goals are translated into objectives
Goals are translated into objectives. Objectives are more concrete statements about desired end states, with time tables, target populations, and resource limits. An objective might be to make the Colorado River safe for swimming and fishing.

Criteria are the measurable dimensions of objectives
Criteria are the measurable dimensions of objectives. Criteria are used to compare how close different proposed policy alternatives will come to meeting the goals of solving the problem. Criteria set the rules to follow in analyzing and comparing different proposed policy alternatives (solutions).

Sample criteria for improving river water quality might be:
effectiveness--how much of an improvement in water quality will this alternative produce? cost--how much will it cost to improve the quality of the river using this alternative? technical--do we have the equipment and know-how to use this alternative? political--is this alternative politically acceptable?

Measures are the actual measurements that will be taken of each proposed policy alternative. For example, measures such as the following might be employed: effectiveness--how many milligrams of pollutants per liter of water will this alternative clean up? cost--how many dollars will be required to implement this alternative? technical--is the necessary equipment for this alternative available and are people trained to use it? political--what percentage of the voting-age population will favor this alternative in a statewide poll?

One difficulty in specifying criteria and measures is that many problem statements have vague, fuzzy, or even conflicting goals. This is often necessary in order to get consensus on taking some action about the problem. But this complicates the selection of criteria.

If dirty rivers are a problem, and the goal is to have clean rivers, what is the most important considerations in choosing between different ways of cleaning up the rivers? Is it cost? Is it effectiveness? Is it equity? What do we mean by "clean"? It is impossible to get rivers 100% clean. Do we use Federal, State, or local standards on admissible levels of toxicity? How will we measure the level of cleanliness that different policy alternatives are likely to produce?

RELIABILITY AND VALIDITY
The criteria and their measures must be unambiguous. They should be relatively straightforward and simple to measure. Their application should produce uniform results, no matter who does the measuring of different alternatives. And repeated measurements of the same alternative should produce the same results, again, no matter who does the measuring.

Criteria and measures should be appropriate to the unit of analysis.
That is, if the goal of a proposed policy alternative is to change the investment strategies of cities, the unit of measurement is cities, not individuals. Be sure to specify whether the unit of measurement is households or families, census tracts or neighborhoods, school children or school districts, etc.

ECONOMIC CRITERIA Most policy analysis involves at least one economic criterion. These include impacts on the economy, on expected public sector revenues, on government spending, etc. The most common economic criteria are costs. These may include: borrowing costs--the costs of borrowing funds decreases in net worth--decreases in assets and/or liabilities direct costs--directly attributable to the policy alternative

indirect costs--additional impacts not included in the goals
intangible costs--costs that cannot be counted or quantified monetarizable costs--can be expressed in dollars one-time fixed costs--new capital expenditures, equipment, training, etc. operations and maintenance costs--ongoing costs of the alternative opportunity costs--other things that could have been done with the same resources instead tangible costs--can be counted and quantified

Costs need to be counted.
One cannot assume that the money was going to be spent anyway. Costs should be identified as completely as possible, eliminating unpleasant surprises down the road. Another type of cost criterion that is often employed is marginal cost. That is, if some good or service is already being produced, how much more will it cost to produce one additional unit of output?

The types of costs that are considered in marginal analysis are:
fixed costs--these do not vary in the short run, no matter how many units are produced variable costs--these vary directly with the volume of output of goods or services average costs--the total of units of output divided by the total costs of output marginal costs--the costs of producing one additional unit of output sunk costs--these are costs that can be ignored as they have already been spent in the past

Another type of economic criterion is benefits.
Benefits are the opposite of costs. Benefits are ways in which the policy actors will be better off. Benefits can be measured in many of the same ways as costs, including: direct benefits--directly attributable to the policy alternative increases in net worth--increases in assets and/or liabilities Indirect benefits--additional benefits not included in the goals

interest earned--interest that will accrue or be paid
intangible benefits--benefits that cannot be counted or quantified monetarizable benefits--can be expressed in dollars one-time benefits--one-time reduction in the problem ongoing benefits--continuing decreases in the problem tangible benefits--can be counted and quantified

Benefits are often more difficult to quantify than costs
Benefits are often more difficult to quantify than costs. One alternative is to use "shadow prices," or the value of the benefits in a perfectly competitive market, for example, free recreation facilities, wilderness areas, parks, etc.

EQUITY CRITERIA Efficiency and effectiveness are technical and economic questions, but equity is a public question. Equity asks about the social allocation of burdens and benefits. Equity asks the questions of "who pays?" and "who benefits?"

A proposed policy alternative may impact equity if it will change the distribution of burdens and benefits in society. There is no universally approved optimal or right answer for how benefits and burdens should be distributed in society. That is a continuing area of contention, and essentially a political decision.

Horizontal equity asks whether burdens and benefits are being shifted among groups in society which are relatively equal. Vertical equity asks whether burdens and benefits are being shifted among groups in society which are relatively unequal. Inter-generational equity asks whether burdens or benefits are being shifted from one time period to another, whether younger generations will have to pay more and receive less than older ones, or vice versa.

Groups are often identified on the basis of:
residence income citizenship race or ethnicity sex age family status home ownership educational status veteran status criminal record substance abuse health

Problems in assessing equity include:
how should the population be sub-divided? how should groups be defined? should historical criteria, the status quo, or desired states be used? what is a burden? what is a benefit? what is a degree of need? what is an ability to pay?

TECHNICAL CRITERIA Effectiveness is often used as a criterion by which to judge policy proposals. Effectiveness is the extent to which the proposed policy will attain the goals set forth in the problem statement. For example, if the goal is to decrease the current teenage driver accident rate, how much will each policy alternative decrease the rate below current levels?

Another technical criterion is technical feasibility.
This asks whether the technology exists or is readily available to implement a proposed alternative. For example, one proposed policy alternative may be to install in all cars a breath analyzing device that would not let a car start if the driver has been drinking. However, this technology is not widely or cheaply available.

Other technical criteria may question whether the measurement of criteria can be conducted at the desired level of reliability and validity. For example, are there tests that can adequately measure whether students in bilingual education programs have the same level of literacy as students in non-bilingual education programs?

POLITICAL CRITERIA Many times the client for the policy analysis will hold a political office. In that case, the policy analyst must often include political criteria in the assessment of proposed policy alternatives. Political viability asks whether or to what extent a proposed policy alternative will be acceptable to relevant powerful groups, decision makers, legislators, administrators, citizens, neighborhoods, unions, or others.

Other ways of assessing political viability include:
acceptability--is the proposed alternative acceptable to policy makers, policy targets, the general public, voters, etc.? appropriateness--is the proposed alternative appropriate to the values of the community, society, the legislature, etc.? Legal--is the proposed alternative legal under current law, or will statutes have to be amended or enacted? responsive--will the proposed alternative meet the real or perceived needs of the target group, the public, etc.?

ADMINISTRATIVE CRITERIA
Many public policies are implemented by public agencies. Therefore, administrative operability or administrative ease are often used as criteria for judging proposed public policies. Questions that may be addressed include: authority--does the agency have the authority to implement the proposed policy?

commitment--does the proposed policy have the commitment of top managers, field staff, and support staff? capacity--does the agency have the resources to implement the proposed policy, in terms of staff, skills, money, training, expertise, etc.? support--are the facilities, equipment, and other support available for the proposed policy?

6. IDENTIFYING ALTERNATIVES
GENERATING ALTERNATIVES Before alternatives can be generated, 1) the problem must be correctly identified, and 2) relevant criteria for judging alternatives must be specified

At first, the policy analyst can generate a large number of alternatives, but later reduce them to a manageable size (between four and seven). Consider alternatives like the status quo, but also radically different. Consider what may be possible under different circumstances.

Some criteria that are often used in judging the suitability of alternatives include:
1) cost--can we afford it; will it be cost-effective? 2) reliability--does it have proven success, or is it subject to failures? 3) stability--will it still work if conditions change? 4) invulnerability--will it work if one of its component parts fails? 5) flexibility--can it accomplish more than one thing? 6) riskiness--is there a high chance of all or nothing?

7) communicability--is it easy to understand?
8) merit--does it address the problem? 9) simplicity--is it easy to implement? 10) compatibility--is it congruent with existing norms and procedures? 11) reversibility--can we return to our prior state if it fails? 12) robustness--can it succeed in different future states?

SOURCES OF ALTERNATIVES
1) The status quo or no action alternative This means that current efforts will continue at the same level. It is important to consider how effective any different alternative will be at changing the status quo. A baseline analysis: identifies clear trade-offs with the present; clarifies project objectives; underlines whether there is a need for action or not; provides linkages to existing efforts; identifies problems likely to emerge; confirms that no optimum solution exists.

2) experiences of others with similar problems, from reported research findings, experts, laws, public opinion polls, new technology, etc. 3) re-define the problem from others' points of view, including opponents of any change 4) consider the ideal, then apply political, economic, and other constraints 5) start from generic, to modified, to custom-made alternatives 6) Quick Surveys by telephone, fax, or , of peers, old MPA classmates, people in the policy issue network, public meetings or hearings, content analysis of editorials, letters to the editor, etc.

7) Literature review of professional and academic journals, government reports, collected proceedings from conferences, on-line services (lexis-nexus, first search, article first etc.). 8) Case studies of real world experiences: why was the alternative adopted, what were the circumstances, what other alternatives were considered and discarded, how did it eventually work out, what modifications were made after implementation.

9) Passive collection and classification: keep a folder for collecting interesting policy solutions on a regular basis, even if no problem exists at the moment, from clients, superiors, advocates, media, interest groups, etc. Then refer to the folder in emergencies. 10) Develop Typologies: identify all the types of persons likely to be affected by any policy alternative, and what the probable reaction of each group would be to each type of alternative suggested; then develop alternatives that can overcome the objections of most of the groups.

11) Use analogies: 'new' problems are really just like other 'old' problems.
Personal Analogy: pretend to be someone affected by this problem, identify with the problem to see what types of policy alternatives suggest themselves; Direct Analogy--look at solutions to other problems to see if they can be applied to this one; Symbolic Analogy--imagine the most aesthetically satisfying solutions rather than merely technologically sound ones; Fantasy Analogy--image the ideal solution, and try to preserve as much of it as possible when working backwards through real world constraints.

12) Brainstorming--can be oral, written, or electronic
12) Brainstorming--can be oral, written, or electronic. Brainstorming has two phases, first a pure idea-generation phase, where no judgments are made about any ideas; and second, an evaluation and ranking phase, to help arrive at concrete solutions. 13) Feasible Manipulation--takes existing policy activities and develops alternatives based on limited, moderate, or wide manipulation of the range of possible activities.

14) Modify existing solutions:
Magnify--do more, more often, larger, longer, exaggerate, add new components, new resources Minify--do less, less often, smaller, shorter, omit, remove, split apart, under use, fewer resources Substitute--switch components, apply in different order, use different materials, try a different location or different sponsor

Combine--blend approaches, combine units, combine purposes, combine sponsors
Re-arrange--reverse, invert, change sequence, speed up, slow down, randomize Location--use single or multiple locations, node versus scattered, temporary versus permanent Timing--accelerate, lag, stagger, run concurrently, shorter span, longer span, time sharing Finance--provide, purchase, tax, user fee, subsidy, co-pay, deductible, voucher, contract out

Organization--centralized versus decentralized, mandated versus voluntary, regulated, prohibited, enforced, inform, implore Decision Sites--individual, unit, organization, elected, appointed, advisory, binding Influence Points--users, providers, intermediaries, beneficiaries, payers Risk Management--guarantees, insurance, remedial correction

PITFALLS 1) Too much reliance on past experience 2) Failure to capture ideas and insights (listen, write them down, record them) 3) Too early closure on problem definition 4) Sets a policy preference too soon before all the alternatives are known 5) Criticizing new ideas as they are offered 6) Some alternatives are ruled out too early on 7) Failure to re-consider discarded alternatives if conditions change

7. ASSESSING POLICY ALTERNATIVES
Which policy alternative should be adopted? In this step in the policy analysis process, the policy analyst takes each of the proposed policy alternatives and, one by one, applies each of the decision criteria to each alternative.

For example, say we have specified that we will be using the criteria of efficiency, cost, political acceptability, and equity to make our decision. We have defined what we mean by each of these and how they will be measured. For example, efficiency is defined as the amount of reduction in the teenage driving accident rate, and is measured by the Department of Motor Vehicles as the number of accident involving teen drivers divided by the total number of teens in the state.

We must then look at each proposed policy alternative, one at a time, and ask,
what would be the efficiency of this alternative? What would be the cost of this alternative? What would be the political acceptability of this alternative? And how will this alternative affect equity? We then repeat this process for every alternative, including the no-action alternative. How do we know what the efficiency of each alternative will be? That involves forecasting.

FORECASTING The criteria that will be important in assessing proposed policy alternatives determine what needs to be forecast. For example, if the goal of a proposed policy alternative is to lower the teenage driving fatality rate, then what needs to be forecast is the teen driving fatality rate, first under the assumption that no action is taken, and the under the assumption that the policy alternative being considered is implemented.

There are a variety of methods used to make forecasts
There are a variety of methods used to make forecasts. Forecasting methods range from simple stereotyping to complex statistical formulas. Intuition may use techniques such as Delphi, scenario writing, or feasibility assessment. However, it requires that the participants be quite knowledgeable, and it needs to be checked for logical consistency.

Theoretical models identify important variables and specify the nature of the linkages among them. Then the model is used to predict outcomes when one or more of the variables are changed. Models are built from information, experience, expert advice, etc. Constructing a model helps to get to the key elements of the situation, and focus on the most important concerns. It identifies the key factors and the relationships among them which will likely be impacted by any proposed policy alternative. It demonstrates the likely consequences of either the no action alternative, or any other rival alternative.

Models may be expressed in words, in physical dimensions (e. g
Models may be expressed in words, in physical dimensions (e.g., architectural models), or in numerical form. Extrapolation uses the past to predict the future, assuming there are stable patterns. For example, if the population of Arizona has been growing at 50% every ten years, then a graph showing past growth can be extended into coming years to predict future growth.

Extrapolation is useful for conducting a baseline analysis, showing what is expected if the status quo or no action alternative is adopted. It is relatively simple and cheap and can be accurate in many circumstances. Data can be either raw numbers or a computed rate of change.

Extrapolation requires precise definitions of criteria and measures, and accurate measurement. It is most often used when there are linear patterns in the data. Extrapolation is less useful in the case of new problems, new issues, or new policy areas, where there is little or no past data.

The most commonly used form of regression is linear regression, and the most common type of linear regression is called ordinary least squares regression. Linear regression uses the values from an existing data set consisting of measurements of the values of two variables, X and Y, to develop a model that is useful for predicting the value of the dependent variable, Y for given values of X.

Elements of a Regression Equation
The regression equation is written as Y = a+bX + e Y= Dependent variable (Y), what is being predicted or explained a=Alpha, a constant; equals the value of Y when the value of X=0 b=Beta, the coefficient of X; the slope of the regression line; how much Y changes for each one-unit change in X. X= Independent variable (X), what is predicting or explaining the value of Y e=The error term; the error in predicting the value of Y, given the value of X (it is not displayed in most regression equations).

For example, say we know what the population has been in past years for a given area. We can compute the values of the components of the regression equation, and, and use them to predict what the area's population will be in future years.

Year Actual Population Predicted Population 1910 1920 1930 1940 1950 1960 1970 1980 1990 2000 2010

If the data are not linear, that is, if on a graph the line that best shows the relationship between the two variables is not a straight line, then simple linear regression cannot be used to extrapolate into the future. Instead, the data must be converted, for example, to logarithms, or a different sort of regression must be used.

ECONOMIC ANALYSIS One of the most widely used economic analysis tools is to look at the long term costs and the long term benefits of a proposed policy alternative. From there, the policy analyst can calculate either the Net Present Value (NPV), the Cost-Benefit Ratio, or the Internal Rate of Return (IRR) of each alternative.

To calculate the long term costs and long term benefits of a proposed policy alternative, the policy analyst must assemble estimates of the initial or implementation year costs and benefits of the alternative, and the subsequent costs and values for each additional year the project will be in effect. Say that the city wants to upgrade its data processing function. It asks for bids showing costs to be submitted by different vendors. It also estimates the savings that will occur from each bid (for example, from a reduction in personnel).

For the first bid, the policy analyst estimates the following
YEAR: 1 2 3 4 5 Costs $15,000 $1,223 Benefits $4,000

Discounting The next step is for the policy analyst to decide on a discount rate. The discount rate assumes that money spent in the future will not cost as much as money spent today. Similarly, money gained in the future will not be worth as much as money gained today. This is based on the human preference for wanting to put off costs (or payments) as long as possible, and wanting to receive benefits (or pay) as soon as possible.

The discount rate is usually obtained from economists, from agency policy, or from the nature of the project being considered (i.e., whether a large infrastructure project, a revenue-bond based project, or a general obligation bond based project). Another source is the discount rate charged by the Federal Bank, or the interest rate paid on government bonds.

At times, the choice of which discount rate to use has been highly politicized. Because many government projects have high initial costs but a long stream of benefits, a low discount rate will make a project look more favorable, and a high discount rate will make a project look less favorable.

The same discount rate is generally applied to both the project costs and the project benefits. If inflation is going to be factored in, it should be applied to both the costs and benefits separately, before the discount factor is applied. To calculate the discounted costs, multiply each year's costs by that year's discount factor (the discount rate factor be obtained from a table of discount rates):

YEAR: 1 2 3 4 5 Total Costs $15,000 $1,223 $16,223 Discount rate 4% Discount factor 1.0 .9615 .9246 .8890 .8548 .8219 Discounted costs $1087 $16,087

To calculate the discounted benefits, multiply each year's benefits by that year's discount factor (the discount rate factor be obtained from a table of discount rates):

YEAR: 1 2 3 4 5 Total Benefits $4,000 $20,000 Discount rate 4% Discount factor 1.0 .9615 .9246 .8890 .8548 .8219 Discounted benefits $3846 $3698 $3556 $3419 $3288 $17,807

Net Present Value The Net Present Value is the value of the project if all the costs were paid today and all the benefits were gained today. To find NPV, subtract discounted costs from discounted benefits: Discounted Benefits $17,807 - Discounted Costs $16,087 = $1,720 The Net Present Value of each policy alternative must be calculated separately, and then it can be compared to the NPV of each other policy alternative, to find the one with the highest NPV.

Cost-Benefit Ratios The costs and the benefits of any policy alternative can be compared in a number of ways. Cost-benefit ratios are obtained by dividing discounted benefits by discounted costs: Discounted benefits =$17,807 Discounted costs =$16,087 Benefit/Cost ratio =1.1

Note that the highest benefit-cost ratio may not have the highest NPV
Note that the highest benefit-cost ratio may not have the highest NPV. These are two different types of analysis. The most efficient projects have the highest benefits-to-costs ratio, but many policy analysts prefer to maximize NPV. In any case, NPV should be a positive number, and the benefit-cost ratio should be greater than 1.0

Internal Rate of Return
The internal rate of return is an expression of the discount rate at which discounted benefits would equal discounted costs. For the example above, at an 8% discount rate, discounted benefits would equal $15,971 and discounted costs would also equal $15,971.

If the calculated IRR is greater than the discount rate being used for the project, then that is an indication that the project should be carried out. Generally, IRR is not comparable to either NPV or the benefits-to-costs ratio. The IRR from one project, however, can be directly compared to the IRR from an alternative project.

Sensitivity Analysis Often there is no clearly superior potential policy alternative, but several that seem equally acceptable. One alternative may be better on the criterion of efficiency, while another is better on costs, and a third on political acceptability. A policy analyst will usually try to see how sensitive the analysis is to changes in assumptions. Things that the policy analyst will test include: 1) the length of the project (how long will benefits continue) 2) the discount rate 3) the value placed on various quantities (costs, benefits, probabilities, etc.)

For example, a city wants to replace old garbage trucks with newer models, and it assumes the new trucks will last 20 years. What if the benefits only last 10 years? Or if the annual maintenance costs are 50% higher than what was budgeted? Or does a project still have a positive NPV if the discount rate is raised from 4% to 6%? Is the IRR still greater than the discount rate? Is the benefit-to-cost ratio still greater than 1.0 ?

In another example, the city assumes that building a new parking garage will raise an additional $2,000 per parking space per year in sales taxes, as well as the revenues from parking. What if only $1,000 is raised? Or say a university wants to get more students to park on campus instead of on nearby neighborhood streets. It thinks that if it reduces the parking permit fee by $10, then 25% more students will buy one and park on campus. What if only 5% more students buy one?

A city has vacant land that it can sell, lease, or keep
A city has vacant land that it can sell, lease, or keep. The city wants to sell. It assumes that someone will buy the land and develop it, increasing the city's property tax revenues. But what if the office building remains mostly vacant? Would this change the city's decision on selling? What are the probabilities of the different outcomes? What if the probabilities for the favored outcome decrease?

Another type of sensitivity analysis is to identify the break-even point. This can vary according to: the length of the project (how many years are needed to break even?) the discount rate (how low before benefits equal costs?) the value of other quantities (e.g., amount of extra parking permits sold?)

Contingency analysis identifies what will happen if one of the basic assumptions about the project is altered. For example, what if there are large cost over-runs? What if people do not behave as predicted (e.g., buy more parking permits?)

A fortiori analysis examines the likelihood that any one factor will take on a value that makes the project infeasible. For example, what if the project takes two years to complete instead of one? What if interest rates rise dramatically? What if new regulations are adopted that make the project technically impossible?

To perform sensitivity analysis,
1) list all relevant considerations; 2) establish the range of values that each variable can take, from low to high; 3) holding all other values constant, vary the value of one variable at a time; 4) test sensitive values to find the break-even, contingency, and a fortiori points.

RISK ANALYSIS Some decision-makers are risk averse. They want to minimize any possible losses, rather than to pursue the (riskier) maximum possible gains. They will want to go for the sure thing (the alternative with the highest probability--in this case, do nothing--especially if limits their possible losses (for this alternative, the worst case scenario is to break even at 0).

Another way to begin the appreciate the different possible outcomes of different policy alternatives is to use quick decision analysis. This is a way to visually represent a small number of alternatives and their consequences.

Quick decision analysis identifies key issues, and helps the policy analyst to decide what information is necessary to assess each possible alternative. It helps to structure thinking about the probability or likelihood that certain outcomes will occur. It also helps the policy analysts or decision-makers to reveal their attitudes about risk and uncertainty. And it alerts the policy analyst to the possible political ramifications of predicted outcomes.

The steps in constructing a quick decision analysis are:
1) identify the dimensions of the analysis (problem, alternatives, outcomes) 2) construct a diagram 3) forecast the likely outcome for each alternative 4) assess how likely each outcome is in terms of probability 5) calculate the expected value of each alternative

For example, say a city wants to know if it should offer tax abatement to encourage economic development. The two alternatives are, simply, to do nothing (not offer the abasement), and to do something (offer the abatement). Each possible policy alternative has two possible outcomes: new economic development occurs, or new economic development does not occur.

Likely Outcomes Do Nothing Offer Abatement Development Occurs No new development Development Occurs No new development Change in Property Tax Revenues +$100 m +$900 m Cost of offering the abatement -$600 m -$200 m Cost of additional city services -$25 m -$100 m Net +$75 m +$200 m Probability of this outcome p=0.3 p=0.7 p=0.6 p=0.4 Expected Value of this alternative 0.3 x $75 m = $22.5 m 0.7 x $0 m = $0 m 0.6 x $200 m = $120 m 0.4 x-$200 m = -$80 m +$22.5 m +$40 m

However, it is important to question quick decision analysis.
-What studies were used to estimate outcomes and probabilities? -Were discount rate applied? -What time frame was considered? -What were the opportunity costs (how could the money be spent elsewhere?) -How sensitive are these figures to changes in the economy? -At what probability would the expected value of the two alternatives be equal?

If there is a great deal of uncertainty about the analysis, there are a number of strategies:
1) delay until more is known 2) map out all uncertainties and the information that is needed 3) collect more data to reduce uncertainty 4) estimate a wide range of possible values for those which are uncertain 5) develop alternatives under a wide range of possible conditions

6) build in more flexibility
7) build in more backup 8) compromise to an acceptable alternative, even if it is not the optimal one 9) choose a strategy that minimizes the maximum possible losses 10) conduct in-depth research to provide the information needed

POLITICAL ANALYSIS Often one criteria for assessing proposed policy alternatives is political acceptability to the client. A political feasibility analysis can help the policy analyst identify the important elements to be considered for each proposed policy. 1) Actors--people, groups, and organizations 2) Beliefs and motivations--which are negotiable, and which are non-negotiable? 3) Resources--power, influence, money, staff, public opinion, etc.

4) Effectiveness--leadership, ability to use resources effectively
5) Sites--agendas, windows of opportunity, sequencing of decisions, etc. A political feasibility analysis takes each proposed policy alternative and examines how well it will hold up in the current political reality. Which actors will favor or oppose it, and why (beliefs and motivations)? What resources do they have, and how effective will they be at supporting or opposing the policy? Where is the debate on the policy to occur, and which actors or groups will be most powerful there? Does any group have veto power?

IMPLEMENTATION ANALYSIS
Even after a policy is adopted, there still may be resistance to its implementation. In conducting an implementation analysis, the policy analyst looks at factors that will make the alternative easier or more difficult to implement, such as: 1) are there few or many actors required to implement this alternative? 2) will there be one or multiple implementation settings?

3) will there be a single or multiple sets of instructions?
4) what is the degree of consensus around this alternative? 5) what magnitude of change will be required? 6) how much of the political conflict from the adoption stage will be displaced into the implementation stage? 7) can game theory be used to model the possible outcomes? 8) are the necessary resources present, such as administrative will, competence, budget, skills, authority, personnel, etc.

8. DECISION RULES and DISPLAYING ALTERNATIVES
Policy assessment techniques do not determine which policy should be adopted. Policy analysis presents the benefits and drawbacks of each alternative, but in addition one or more decision rules are needed in order to determine which policy is the "best."

There are many problems in trying to determine which policy to adopt.
1) Many problems in the public sector have multiple facets. Policies are designed with multiple goals or objectives. There may be no dominant objective, or several objectives may be in conflict. 2) there are multiple criteria to take into account--technical, economic, political, and administrative--but who decides which is the most important? 3) not all important considerations can be converted into comparable units, such as dollar values. 4) which is the proper criterion to use, greatest net present value? greatest internal rate of return? largest benefit-cost ratio?

5) there is often a lack of agreement beforehand on decision rules, or which rules to apply
6) even if each decision criterion is optimized separately, there may still be a sub-optimal choice at the end (a camel is a horse designed by a committee). The policy analyst is often faced with trying to present multiple policy alternatives which have been assessed in terms of multiple decision criteria. There are various methods which can be used to display this information in a way that facilitates decision-making.

PAIRED COMPARISONS If there are 5 different ways of collecting the city's garbage, then the analyst can compare method 1 with method 2 and determine which is superior. The better of the two is then compared with method 3, and again one of them is determined to be superior. The winner of each contest is then compared with another of the remaining alternatives until all have been evaluated and the winner of the last contest is the overall winner.

This method presents a simple, step-by-step comparison that is relatively easy to follow, if somewhat tedious. The problem with this method is that the outcome may be influenced by the order in which the alternatives were considered.

SATISFICING The analyst presents all the alternatives that meet the minimum threshold levels on all criteria. The minimum threshold is then increased on each criterion, and those alternatives which do not meet the new levels are dropped. This process continues until only one alternative is left.

For example, say there are 6 alternatives to solve the parking problems on campus and three criteria: net revenues, number of permits sold, and student satisfaction. The minimum levels are $50,000 in revenues, 10% increase in permits sold, and at least 50% students satisfaction. Two alternatives are dropped because they do not meet one or more of these minimum levels.

The levels are then raised to $75,000 in revenues, 20% increase in parking, and 65% student satisfaction. Another two alternatives drop out. Finally, raising the minimums to $100,000 in revenues, 25% increase in parking, and 75% student satisfaction eliminates one alternative and the only remaining alternative that meets all these minimums is the winner.

This method assures that the minimum "needs" for the policy will be met, and, in addition, offers the prospect of meeting higher than minimum needs ("desires"). The problem is that it changes the definition of acceptable after the analysis has been completed.

GRADING METHOD The consequences of each alternative on each criterion are considered. A grade of "Pass" or "Fail" is assigned to each alternative on each criterion. Only those alternatives which "Pass" on all criteria are retained; those which have any "Fails" are rejected. The retained criteria are then compared further. This is comparable to the satisficing method discussed above.

LEXICOGRAPHIC ORDERING
The analyst lists all those alternatives that ranked most highly on the one most important criterion. These alternatives are all considered to be roughly equal. The alternatives are then compared on the second most important criterion. Those alternatives which rank most highly are retained, and they are then compared on the third most important criterion. This process is repeated until the alternative that is most highly ranked on all the criteria is found.

For example, say there are 5 alternatives for economic development for the downtown area, and there are three criteria: increased revenues, increased jobs, and citizen satisfaction. Three alternatives rank highly on the most important criterion of increased jobs. These three are then compared on the second criterion, citizen satisfaction. Only two are highly ranked. These last two are compared on the third criterion of increased revenues, and the most highly ranked is the winner.

This is a rather straightforward process of comparison
This is a rather straightforward process of comparison. However, it assumes there is agreement on which is the first most important criterion, the second most important, and so on. For situations with multiple objectives (criteria) or multiple decision makers, this may be difficult.

NON-DOMINATED ALTERNATIVES
All alternatives are measured on all criteria, and their rank order on each criterion is displayed. Every alternative that is ranked the most highly on any criterion is retained; any alternative that is not ranked #1 on at least one criterion is discarded. The result is only "non-dominated" alternatives, that is, alternatives that are clearly superior on at least one criterion. These alternatives can then be compared further by another method.

Criterion 1 Criterion 2 Criterion 3 Alternative A Rank #4 Rank #5 Rank #6 Alternative B Rank #2 Rank #1 Rank #3 Alternative C Alternative D Alternative E Alternative F

In this example, only Alternatives B, C, and D were ranked #1 in at least one category. Alternative B is not dominated by any other alternative on Criterion 2; Alternative C is not dominated by any other alternative on Criterion 3; and Alternative D is not dominated by any other alternative on Criterion 1. Alternatives B, C, and D must now be considered further.

Say there are five designs for a new community building, and the designs are ranked in terms of their suitability for athletics, as well as their suitability for arts and crafts. Only one design will be ranked #1 in each category. These two designs are the only non-dominated alternatives. Since they are each ranked #1 on one category, and they are each ranked #2 in the other category, they are considered equal.

A third criterion may be added as a "tie-breaker
A third criterion may be added as a "tie-breaker." For example, the two alternatives might be ranked in terms of their suitability for meetings. The higher ranking of the two will be the winner.

This method is straightforward and relatively easy to follow
This method is straightforward and relatively easy to follow. However, it brings in additional criteria (tie-breakers) after the analysis has been completed and some alternatives have been eliminated.

EQUIVALENT ALTERNATIVES
When at least one of the criteria can be measured in quantitative units, for example, dollars, this method can be used to compare two alternatives. It involves converting other units of measurement to dollars as well and then comparing the two alternatives again.

For example, you have been offered two possible jobs
For example, you have been offered two possible jobs. You assess these jobs in terms of five criteria: salary (measured in dollars), climate (measured in days of sunshine per year), commute time (measured in minutes), nature of job (interesting or uninteresting), and potential for advancement (measured as good or poor).

Salary Climate Commute Nature Advance Job A $36,000 240 20 I G Job B $42,000 200 30 U P

Since the two jobs are both measured on at least one criterion--salary--in a quantitative form (dollars), the next step is to convert the value of the other criteria to dollars as well. For example, how much of the $42,000 salary would you be willing to give up to have more days of sunshine per year? Say that the extra 40 days of sunshine per year are worth $1,600 (you would be willing to take Job B at $1,600 less in pay if it had 40 more days of sunshine). Show these calculations in a revised chart.

Salary Climate Commute Nature Advance Job A $36,000 240 30 I G Job B $40,400 20 U P

Now, how much of the remaining salary of Job B would you be willing to give up to have a shorter commute? Say that cutting commute time by 10 minutes per day is worth $1,000 (you would be willing to take Job B at $1,000 less in pay if it had 10 minutes less of commuting). Show these calculations in a revised chart.

Salary Climate Commute Nature Advance Job A $36,000 240 20 I G Job B $39,400 U P

Now, how much of the remaining salary of Job B would you be willing to give up to have a more interesting job? Say that it is worth $1,400 (you would be willing to take Job B at $1,400 less in pay if it was more interesting). Show these calculations in a revised chart.

Salary Climate Commute Nature Advance Job A $36,000 240 20 I G Job B $38,000 P

Now, how much of the remaining salary of Job B would you be willing to give up to have greater potential for advancement? Say that it is worth $2,500 (you would be willing to take Job B at $2,500 less if there was greater advancement potential). Show these calculations in a revised chart.

Salary Climate Commute Nature Advance Job A $36,000 240 20 I G Job B $35,500

Now the two jobs are "equalized" on all criteria except the quantitative on (salary). Since the two jobs are equal except for salary, then the job with the higher salary (Job A) is the winner.

This method is rather complex and has many steps
This method is rather complex and has many steps. It is also very subjective and best suited to situations where there are only individual decision makers rather than groups. It does help the decision maker to clarify their personal preferences, however, and may provide useful insight into the most important facets of the problem.

WEIGHTED DECISION CRITERIA
When there are multiple decision criteria, a ranking or weighting system can be developed to reflect the relative importance of each criterion in the decision making process. These are sometimes referred to as "importance weights."

Say that the state wants to adopt a plan to give grants to the elderly to defray their utility expenses. The decision criteria are number of elderly reached, speed at delivering the grants, and controls on fraud. Each alternative is measured on each criterion and a raw score is assigned.

Elderly Reached: 5 = 80% - 100%; 4 = 60% - 80%; 3 = 40% - 60%; 2 = 20% - 40%; 1 = 0% - 20% Speed of Delivery: 5 = 0-5 days; 4 = 6-10 days; 3 = days; 2 = days; 1 = 21+ days Curbs on Fraud: 5 = Very strong; 4 = Strong; 3 = Moderate; 2 = Weak; 1 = Very Weak

RAW SCORES Elderly Reached Speed of Delivery Fraud Curbs Alternative A 5 4 3 Alternative B Alternative C Alternative D 2

The decision criteria have been weighted by the decision makers, so that the number of elderly reached is worth 50% of the decision, the speed at delivering grants is worth 30%, and controls on fraud is worth 20%.

The raw score for each alternative on each criterion is then multiplied by the importance weight for each criterion. The result is a weighted score. The weighted scores for each alternative on all the criteria are then added for the total weighted score.

WEIGHTED SCORES Elderly Reached (50%) Speed of Delivery (30%) Fraud Curbs (20%) Total Score Alternative A 5 x .5 = 2.5 4 x .3 = 1.2 3 x .2 = 0.6 4.3 Alternative B 4 x .5 = 2.0 4 x .2 = 0.8 4.0 Alternative C 3 x .3 = 0.9 3.5 Alternative D 2 x .5 = 1.0 5 x .2 = 1.0 2.9

The alternative which scores the highest on the number of elderly reached (Alternative A) is the alternative with the highest total score. An alternative which has very good controls on fraud, but does not do as well in the areas of reaching the elderly or speed, has the lowest total score.

The advantages of this method are that arriving at the superior alternative is straightforward and relatively easy. Rather than considering alternatives one at a time, or considering criteria one at a time, this method considers all alternatives on all criteria simultaneously. The disadvantages are that many criteria are not measurable in quantitative form and there may be disagreement over the importance weights of the criteria.

GROLLER SCORECARD The matrix (or "scorecard") is a way of displaying the impacts of each alternative in terms of each criterion in "natural" units of measure, such as reduction in fatality rate, political acceptability, administrative ease, effectiveness, impact on need for additional landfills, etc. The scorecard indicates the extent to which each alternative attains the objectives specified by each criterion.

This approach the viewer(s) to grasp a complex analysis by summarizing it in a chart. It can help a group to come to a decision without the need to impose quantitative measurement or assign importance weights to criteria.

REDUCE PERMIT PRICE STRONGER PARKING ENFORCEMENT EDUCATIONAL CAMPAIGN COST TO THE UNIVERSITY More students buy permits; no net loss or gain Higher costs in enforcement Moderate costs for materials SOLVES PARKING PROBLEM Cuts illegal parking by 25% Cuts illegal parking by 40% Cuts illegal parking by 10% ACCEPTABILITY TO STUDENTS Highly acceptable Highly unacceptable Moderately acceptable ACCEPTABILITY TO COMMUNITY Unacceptable Best option Worst option

9. IMPLEMENTATION, MONITORING, & EVALUATION
IMPLEMENTATION ANALYSIS The full policy process is often described by the following steps: 1) problem definition 2) alternative generation 3) analysis of alternatives 4) policy adoption 5) policy implementation 6) policy evaluation

While this course has focused on the first three steps, the last three steps are equally important. A thorough policy analysis will include some consideration of policy implementation, monitoring, and evaluation.

The policy analyst can sketch out an implementation plan for the most highly ranked alternative(s) that considers: 1) relevant actors and their interests 2) required resources and who might provide them 3) facilitators and barriers likely to be encountered 4) reasonable time frame

Implementation analysis might involve writing a "best-case" scenario and a "worst-case" scenario for each policy alternative, as well as the "most likely" outcome. The idea is to think systematically through the implementation process, identify potential problems, and develop actions that can be taken to either avert catastrophes or reduce losses.

POLICY MONITORING Policy maintenance refers to keeping the policy or program going after it is adopted. Policy monitoring refers to the process of detecting how the policy is doing.

To monitor a policy, some data about the policy must be obtained
To monitor a policy, some data about the policy must be obtained. A good implementation plan will suggest some ways in which ongoing data about the policy can be generated in the regular course of policy maintenance, for example, from records, documents, feedback from program clients, diary entries of staff, ratings by peers, tests, observation, and physical evidence.

POLICY EVALUATION Policy evaluation is the last step in the policy process. It may ask deep and wide-ranging questions, such as: 1) was the problem correctly identified, or was the correct problem identified? 2) were any important aspects overlooked? 3) were any important data left out of the analysis? did this influence the analysis? 4) were recommendations properly implemented? 5) is the policy having the desired effect? 6) are there any needs for modification, change, or re-design? what should be done differently next time?

When policies fail to have the intended effect, it is usually due to one of two types of failure: theory failure, or program failure. A theory failure occurs when the policy was implemented as intended, but failed to have the desired effect. This may occur when, for example, a school adopts school uniforms to curb violence in the school, but the violence remains at the same level.

The policy was implemented (uniforms were adopted) but the expected change did not occur. The theory that violence occurs due to style of dress is wrong. There must be some other cause of school violence, which would require a different policy to address.

An implementation failure occurs when the policy is not implemented as intended. For example, the school may adopt a uniform policy, but the majority of the students ignore it. The level of violence in the school does not change. We still do not know whether adopting school uniforms would lower the level of violence in the schools; we only know that uniforms were not adopted.

FORMATIVE EVALUATION If adequate monitoring processes are in effect, it should be fairly easy to detect whether a policy has been implemented as intended. This type of policy monitoring has been referred to as formative evaluation. Formative evaluation documents and analyzes how a policy is implemented, with the objective of making improvements as the implementation process unfolds.

SUMMATIVE EVALUATION Summative evaluation is conducted after a program has been fully implemented. It looks at whether the program is meeting its objectives, and why or why not.

Evaluations may be unpopular for many reasons:
1) the program is controversial; 2) there are strong political interests in seeing it succeed or fail; 3) there are difficulties in measuring program accomplishments; 4) those involved may be uncooperative; 5) program effects may be influenced by outside developments.

To decide whether an evaluation will be helpful, the answer to the following questions should be "yes": 1) will the evaluation be accepted by politicians, administrators, and/or participants? 2) has an evaluator been involved from the beginning? 3) are there measurable objectives? 4) are data available? 5) are multiple evaluation methods plausible? 6) has the program remained stable over time? 7) can program staff become involved in the evaluation? 8) will the findings be made widely available?

EVALUATION DESIGN Policy evaluation applies accepted social science research methods to public programs. The same research designs used in laboratory experiments are not always practicable in the field, but the same principles can guide the planning and execution of policy evaluation.

Before-and-After Evaluation: a policy is evaluated for the changes it has produced since its implementation; the situation is controlled to exclude other possible influences on the outcome. With-and-Without Evaluation: a policy is evaluated for producing changes in the target population, compared to another population without the policy.

After-Only Evaluation: the extent to which the policy goals were achieved, compared to the state of affairs before the policy was implemented; but the situation is not controlled to exclude other possible influences on the outcome. Time-Series Evaluation: the changes produced by the policy, tracked over a long time period.

INTRODUCTION TO POLICY ANALYSIS Text:

Similar presentations

Presentation on theme: "INTRODUCTION TO POLICY ANALYSIS Text:"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

INTRODUCTION TO POLICY ANALYSIS Text:

Similar presentations

Presentation on theme: "INTRODUCTION TO POLICY ANALYSIS Text:"— Presentation transcript:

Similar presentations

About project

Feedback