Back to Basics with PR Evaluation ‘w alk in, dance out’ Paul Noble Partner, Noble Ink

I have begun to feel more and more like a fundamentalist preacher railing against sin; the difference being that I have railed for evaluation in public relations practice; just as everyone is against sin, so most public relations people I talk to are for evaluation. People keep on sinning, however, and PR people continue not to do evaluation research. Grunig

Why Evaluation is Not Done Evaluation is seen as expensive Time spent on evaluation could be spent on something else Evaluation is only seen as a post campaign activity Evaluation requires research skills Selling Paul the toothpaste PR people are frightened of the results of evaluation

Recommendations to Practice  Use a more accurate description than ‘PR ROI’  Cost of evaluation to be considered against the business case of what PR can be achieved  Culture of the PR industry should shift towards more sophisticated measurement  Many problems stem from a view that ‘PR is basically free advertising’  Place more emphasis on the technical understanding required to conduct proper PRE CIPR/CDF

PRE Shortcomings  Failure to tap into existing sources of research, or to develop a relationship with market research providers  Failure to align communications objectives with organisational goals  Tendency to go into creative mode before planning  Using terminology for effect rather than clarity  Focussing too heavily on the value of media publicity Fairchild

Objectives SMART objectives (the ideal) Process objectives (an oxymoron) Hierarchy of objectives: awareness, attitudes, behaviour

Advertising Equivalency A means of converting editorial space in the media into advertising costs, by measuring the amount of editorial coverage and then calculating what it would have cost to buy that space. Most reputable researchers contend that advertising equivalency computations are of questionable validity, … … continue to be used despite being completely discredited CIPR/CDF (2004) Do not use Advertising Value Equivalents (AVEs)

Opportunities to See Average number of exposures experienced by the audience covered by a particular medium An advantage of OTS is that most channels have an OTS figure, but they are measured very differently It means that the message is available to the target, who may or may not choose to attend to it OTS is more of a planning than an evaluation tool

Research and PR Research to plan programmes Research to monitor efficiency Research to assess the impact The power of dirty data Triangulation

The Substitution Game The most common error in program evaluation is substituting measures from one level for those at another level. … if asked to document program impact, they substitute publicity placements, …, for the changes … spelled out in the program objectives. Evaluation researchers refer to this as the “substitution game”. Cutlip et al

Case Studies Westminster City Council: Bin There Done That National Patient Safety Agency: Raising Awareness and Changing Attitudes Woking Borough Council: Triangulation in Practice

Conclusion  There is no simple solution  Evaluation starts with effective objective setting  Evaluation is a research-based discipline  Evaluation focuses on the PR process  Evaluation focuses on the impact of PR  Evaluation is realistic because PR is pragmatic