1. National Public Health Institute, Finland www.ktl.fi Open Risk Assessment Lecture 2: General assessment framework Mikko Pohjola KTL, Finland

2. National Public Health Institute, Finland www.ktl.fi Contents General assessment framework Purpose of assessments Societal context of assessments Properties of good assessments Universal assessment products General assessment processes Participation in assessments Summary

3. National Public Health Institute, Finland www.ktl.fi General assessment framework A general framework for making all kinds of assessments about any part of reality –HIA, RA, CBA, CEA, IA, … –Currently focus mainly in environmental health Description of general aspects shared by all kinds of assessments –Context –Assessment process –Assessment products

4. National Public Health Institute, Finland www.ktl.fi General assessment framework

5. National Public Health Institute, Finland www.ktl.fi General assessment framework Processes and products are important to be understood and described in detail Context is often not necessary to be described explicitly, but the implications to assessments (products and processes) are

6. National Public Health Institute, Finland www.ktl.fi Purpose of assessments Assessments are descriptions of reality –(or making descriptions of reality) Ultimate purpose of assessment is to improve societal decision making Two aspects of the purpose: –General purpose: to describe reality –Instrumental purpose: to answer to a specific need

7. National Public Health Institute, Finland www.ktl.fi Purpose of assessments The reality is described as variables –Independent information objects, given their scope –Conscious distortion of the description not acceptable An assessment is structured according to a specific need –The specific purpose drives the assessment The specific purpose of assessment must be explicitly defined –Information processing for intended use/user

8. National Public Health Institute, Finland www.ktl.fi Societal context of assessments An assessment is collection, synthesis and interpretation of scientific information and value judgments for use of the society

9. National Public Health Institute, Finland www.ktl.fi Societal context of assessments Assessment is interplay between several different people/groups having different views and roles –Assessors – professional role of producing information –Decision makers – professional role of using the information in decision making –Stakeholders – often unprofessional role of guarding own or other’s interests Experts or decision makers not involved in a particular assessment can also be stakeholders

10. National Public Health Institute, Finland www.ktl.fi Universal assessment products Assessment products are information Assessment products compose of different kinds of objects: assessments and variables In addition,assessments and variables can be grouped into: classes Three universal product object types: –Assessment –Variable –Class

11. National Public Health Institute, Finland www.ktl.fi Universal assessment products Variable –Description of reality within a given scope –Basic building block of assessments –A hypothesis of a state of being Assessment –A compilation of variables belonging to an assessment –Also assessment-specific information –Information for a specific need Class –A collection of objects sharing a property or properties Example: Beneris fish case

12. National Public Health Institute, Finland www.ktl.fi Universal assessment products Causal diagram is a practical means of presenting assessments:

13. National Public Health Institute, Finland www.ktl.fi General assessment processes Observation of reality – Data production Information processing –Collection –Manipulation –Synthesis Management of assessment process

14. National Public Health Institute, Finland www.ktl.fi General assessment processes

15. National Public Health Institute, Finland www.ktl.fi General assessment processes Information collection –Bringing unstructured information about reality from the context to within the sphere of assessment –E.g. data searches, stakeholder meetings, questionnaires, browsing articles, expert elicitation, … Information manipulation –Operating with unstructured information within the sphere of assessment, but not in a structured format –E.g. running models, making calculations, data summarizing, …

16. National Public Health Institute, Finland www.ktl.fi General assessment processes Information synthesis –Converting unstructured information to structured information (i.e. defining variables, assessments and classes) –Combining information with other information in the assessment product Assessment process management –Organizing and aiding the above-mentioned sub- processes –Tools: automated management

17. National Public Health Institute, Finland www.ktl.fi Properties of good assessments

18. National Public Health Institute, Finland www.ktl.fi Properties of good assessments Quality of content –How good description of the given part of reality a particular assessment is? Applicability –How well does the information within the description transfer to its intended use process (decision making)? Efficiency –(Quality of content + Applicability)/Effort –How good output, given the effort, is/was produced?

19. National Public Health Institute, Finland www.ktl.fi Properties of good assessments The basis for evaluating assessments Quality of content – assessment product Applicability – both assessment product and process –Assessment process use process interaction Efficiency – assessment process

20. National Public Health Institute, Finland www.ktl.fi Participation in assessments Groups participate, share information, and utilise the products of others –Common target of work –Group communication & production –Independent actors – Open participation: anyone, anytime, anything Dimensions of openness –Involvement, access, influence, time, …

21. National Public Health Institute, Finland www.ktl.fi Participation in assessments A multifaceted example of information processing –Collection: Several participants bringing their knowledge, information and views to the assessment –Manipulation: Explicit manipulation of information from others, e.g. summarizing Implicit manipulation of information from others taking place in brain –Synthesis: Defining assessments and variables and and refining their descriptions –Management: Meetings, virtual collaboration, resource allocation, task distribution, …

22. National Public Health Institute, Finland www.ktl.fi Participation in assessments Normative & ethical reasons –E.g. legal requirement of participation, Aarhus convention Instrumental & pragmatic reasons –E.g. better acceptance of outcome by those involved Epistemological & substantive reasons –Improved knowledge and output E.g. more points of view, more sources of information, higher relevance by better scoping, …

23. National Public Health Institute, Finland www.ktl.fi Participation in assessments Assessors, decision makers and stakeholders all participate in joint knowledge production Goal: improved output of assessment –Quality of content -> relevance –Applicability -> acceptability –Efficiency (?) -> inter-assessment efficiency

24. National Public Health Institute, Finland www.ktl.fi Summary Assessment –General assessment processes Participation in assessments –Universal assessment products Context –Purpose of assessments (use process) –Properties of good assessments (use process) –Different actors and roles in assessments

25. National Public Health Institute, Finland www.ktl.fi Other important concepts See: Heande Glossary –(http://heande.pyrkilo.fi/heande/index.php/Glossary)