IAEA International Atomic Energy Agency

IAEA Outline PART I General review principles Team work Daily team meetings Counterpart and reviewer involvement in data collection PART II Reviewing additional written material Taking notes Interviews Direct observations Site visits Basic IRRS Training - Methodology of the Review2

IAEA Learning objectives After finishing this lecture the trainee will understand: The general review principles behind an IRRS mission How team work is organized and conducted in IRRS The purpose, conduct and output of Daily Team Meetings The roles of reviewers and counterparts in data collection The role, methodology and contribution of different data collection methods in IRRS: review of additional documentation, note taking, interviews, direct observation, site visits Basic IRRS Training - Methodology of the Review3

IAEA The Great Challenge The IRRS team has, in a short time and in a new environment, to:  look for issues  identify findings  agree findings with counterpart  write a report This Training Lecture provides recommendations on how to work in order to accomplish these tasks in the best way Basic IRRS Training - Methodology of the Review 4

IAEA Detailed contents of Part I 1.General review principles (1-6) 2.IRRS Team Goal 3.Team work, organization (1-3) 4.Team work, methodology (1-3) 5.Team work, expectations (1-3) 6.Daily team meetings, purpose 7.Daily team meetings, agenda 8.Daily team meetings, conduct (1-4) 9.Counterpart involvement in data collection (1-2) 10.Reviewer involvement in data collection (1-2) Basic IRRS Training - Methodology of the Review5

IAEA GENERAL REVIEW PRINCIPLES Basic IRRS Training - Methodology of the Review6

IAEA General review principles (1) To be well prepared, the reviewer should:  Apply common regulatory knowledge and experience  Use the IRRS Guidelines, latest edition 2013  Use the applicable IAEA Safety Standards  Use relevant parts of the ARM including the self- assessment and the action plan Basic IRRS Training - Methodology of the Review7

IAEA General review principles (2) To be efficient, the reviewer should:  Maintain a creatively inquisitive attitude  Keep an open mind  Take and keep copious notes  Gather as many facts as possible to support an issue  Look for examples of good and bad practices  Look for examples to confirm the issues  Let the facts lead to issues Basic IRRS Training - Methodology of the Review8

IAEA General review principles (3) To be professional, the reviewer should:  Maintain a strong focus on the results achieved by the regulatory body the performance of the regulatory staff the regulatory programmes and processes  Reach own conclusions based on facts  Apply the ”so what test” (is the finding really so important for regulatory effectiveness and efficiency?) Basic IRRS Training - Methodology of the Review9

IAEA General review principles (4) To be a skilled reviewer, the team member should:  Bring concerns with the review process to the Team Leader´s attention early  Make effective use of time (there are many issues to deal with, time goes quickly)  Adhere to local rules and act as a good representative of the IAEA  Be a good team worker (be receptive, use support offered by the team, seek advice and support from other reviewers, provide helpful information to others and involve team early in conclusions) Basic IRRS Training - Methodology of the Review10

IAEA General review principles (5) Remember:  As a reviewer you always have to be flexible and receptive  Recognize that there are several ways to accomplish a certain objective or task  Do not try to impose your own solutions on the counterpart but give examples if necessary to illustrate an issue  A relaxed interview situation is best for all parties  You and your counterpart are colleagues, not competitors or adversaries, and should share the same interest in having a constructive peer-review Basic IRRS Training - Methodology of the Review11

IAEA General review principles (6) Thus you have to be Well prepared Efficient Professional Skilled Flexible Collegial Basic IRRS Training - Methodology of the Review12

IAEA TEAM WORK Basic IRRS Training - Methodology of the Review13

IAEA IRRS Team Goal What are we here to do? A joint search for improvement of nuclear regulation through providing recommendations and suggestions, and to illustrate good practices  Review against the IAEA Safety Standards and best international practice by reviewing documents interviewing observing and  Write a report consistent with the IRRS Template, and all in a sprit of good teamwork Basic IRRS Training - Methodology of the Review14

IAEA Team work, organization (1) Basic IRRS Training - Methodology of the Review15 The Standard Team Assignment Scheme assigns reviewers to modules

IAEA Team work, organization (2)  In order to use reviewers efficiently and to provide the necessary communication in the review process, reviewers are assigned to groups working with more than one module  The number of reviewers in each group depends on the size of the nuclear programme to be reviewed  A Group Leader coordinates the work of each group and acts as spokesperson for the group  The Team Leader and the IAEA Coordinator fully concentrate on leading the team and do not take part in the reviews  The Deputy Team Leader normally supports Group A  The IAEA Deputy Coordinator is assigned to a group Basic IRRS Training - Methodology of the Review16

IAEA Team work, organization (3) Basic IRRS Training - Methodology of the Review17

IAEA Team work, methodology (1) An IRRS mission can not succeed without good team work. Good team workers:  Support and help other team members  Seek guidance or assistance from other team members  Use available good English speakers for language review  Accept and use critical comments Do not be defensive Critical comments from the TL/DTL should be understood as a challenge to the issue, not a criticism of the team member´s work Basic IRRS Training - Methodology of the Review18

IAEA Team work, methodology (2) Seek and accept assistance or guidance from the TL or DTL early with respect to:  Problems with counterparts  Problems with interpreters  Difficulties in writing working notes  Personal problems  Problems with team activities or relationships  Let the team leader know if your counterpart invites you home or out Basic IRRS Training - Methodology of the Review19

IAEA Team work, methodology (3) Indications of a good team work: when team members  Feel responsible for the mission as a whole  Alert the assigned reviewer and team to potential weaknesses and good practices in areas other than his/her own  Demonstrate ownership of issues in own area found by others  Are proactive in discussing common boundaries with other reviewers  Respond timely to requests for collecting facts to support a potential issue in a different area  Demonstrate active interest in all areas during the team meetings Basic IRRS Training - Methodology of the Review20

IAEA Team work, expectations (1) 1)Be open minded and critical in thought, objective in discussions and writing 2)Work hard: Evenings are work time Write and update working notes Define your issues Prepare for the next day 3)Relax on free Sunday and enjoy the social programme 4)Prioritize your effort You can not cover everything in IRRS Guidelines to the same depth Use discussions and observations to identify performance and importance Basic IRRS Training - Methodology of the Review21

IAEA Team work, expectations (2) 5)Use IAEA Safety Standards The summary reports of the IRRS findings as reference material 6)Prepare well for team meetings Focus on potential issues Respect the time of others 7)Stay on schedule Review work - daily Working notes writing - daily Be a team worker, help other reviewers! The Team Leaders are there to support you all! Basic IRRS Training - Methodology of the Review22

IAEA Team work, expectations (3) You are all there as representatives of the IAEA:  Respect the culture of the host country  Have security badges and dosimeter (site visits) visible  Follow industrial safety (site visits) and security rules  Don´t take any documents without permission  Be escorted in premises outside dedicated IRRS space  Conduct discussions professionally without display of emotions  Set a good professional example Basic IRRS Training - Methodology of the Review23

IAEA DAILY TEAM MEETINGS Basic IRRS Training - Methodology of the Review24

IAEA Daily team meetings, purpose Meeting of the whole team and the host country Liaison Officer at the end of each working day, until the first draft of the mission report has been completed. Its purpose is  To discuss the main observations of the day  To share findings and insights regarding technical and policy issues related to the different review areas Daily team meetings are not meant for general activity reporting by reviewers  The team leader has to establish an effective style and conduct of these meetings to prevent wasting reviewer´s time meant to work on their reports Basic IRRS Training - Methodology of the Review25

IAEA Daily team meetings, agenda  Highlight the day´s key observations in each review area, particularly significant concerns or positive features which may form the basis for recommendations, suggestions or good practices  Report issues which need to be brought to the attention of other reviewers, especially issues that have a bearing on the remainder of the mission  Identify gaps, overlaps and areas where the information is not clear or is inconsistent  Determine whether any of the day´s observations might affect the remaining schedule for the mission  Summarize the visits/interviews to be conducted during the next day to enable all team members to provide inputs  Determine the status of each reviewers written input to the draft mission report Basic IRRS Training - Methodology of the Review26

IAEA Daily team meetings, conduct (1)  Team members and review groups need to be prepared for the meeting  Use a daily report form or observation notes  Meetings start promptly, normally at 17:00 h and should normally not exceed one hour  All team members are required to attend  Each Review Group Leader has 5 min to inform on key issues in his/her area  Findings, insights and concerns only, no general activity report, avoid descriptive material  Team members are encouraged to participate with relevant contribution, without unnecessarily prolonging the meeting Basic IRRS Training - Methodology of the Review27

IAEA Daily team meetings, conduct (2) Team Leader to standardize the team meetings, TL or DTL to lead the meetings in a strict way  Interrupt presentations exceeding 5 min  Ask for any additional comments from Team Members regarding each presentation  After all presentations, summarize: Preliminary conclusions Concerns to be further addressed Directions for the further work, such as changes in the mission sched ule  Let the Liaison Officer answer questions and to inform about any changes in counterpart´s activities and plans Basic IRRS Training - Methodology of the Review28

IAEA Daily team meetings, conduct (3) Common mistakes:  The meeting has no standardized structure  The TL asks around the table for the day´s impressions  The team members have not prepared to present conclusions and concerns only, but provide a general report about the activities of the day  The TL does not limit the time for presentations and discussions, the meeting can take several hours  The TL does not provide clear conclusions and directions for the further work  The Liaison Officer interferes in discussions in a defensive way rather than answering questions and providing clarifications Basic IRRS Training - Methodology of the Review29

IAEA Daily team meetings, conduct (4) Summary of Daily Team Meetings Is meant to summarize daily observations and cross-cutting issues, it is not an activity report Team members need to thoroughly prepare the report to the meeting Group-wise Both the Group-reports and the entire meeting need to be limited in time Should be closed with conclusions and instructions for the next day Basic IRRS Training - Methodology of the Review30

IAEA COUNTERPART AND REVIEWER INVOLVEMENT IN DATA COLLECTION Basic IRRS Training - Methodology of the Review31

IAEA Counterpart involvement (1) To gain the counterpart´s trust and support, the reviewer should:  Establish a productive, cooperative relationship early  Encourage the counterpart to develop equal responsibility for the quality of the review - it is not in the host organization´s interest to have a low quality report Basic IRRS Training - Methodology of the Review32

IAEA Counterpart involvement (2) The reviewer should encourage the counterpart to:  Actively contribute (but reviewer must beware of not being used in internal conflicts of interest)  Help obtain correct and complete information so that the reviewer has a good understanding (make sure all facts are on the table, check interpretation of facts with counterpart)  Assist in the search for improvement (assess emerging issues)  Develop a sense of ownership of any issue (counterpart can point out significant aspects and help to formulate issues to fit the corporate culture)  Keep an open minded approach (encourage counterpart not to be defensive, remember IRRS is reviewing management processes and not individual performance) Basic IRRS Training - Methodology of the Review33

IAEA Reviewer involvement (1) The reviewer needs to  Make preparations (prepare lists of topics to be covered and questions to be asked)  Control the length of host organization´s presentations (should be short overviews focussed on the topics to be discussed, it is not uncommon that interview time is consumed by long presentations of material already included in the ARM)  Ask questions for clarification (don´t assume, ask questions)  Concentrate on important aspects (focus on key issues, don´t get trapped in detail, avoid interesting discussions which do not contribute to the review at hand)  Keep copious notes Basic IRRS Training - Methodology of the Review34

IAEA Reviewer involvement (2) The reviewer also needs to:  Check related documentation (if gaps ask for any other records, documentation is an important source of facts- use it)  Check processes or records  Examine the distribution process ( who gets it, what do they do, revision process, are documents current)  Check for common understanding (ask at different levels and organizational units how the procedure is understood and used)  Check for customer satisfaction (are outputs accepted)  Check any performance statistics/indicators (can indicate areas and trends to investigate further) Basic IRRS Training - Methodology of the Review35

IAEA Basic IRRS Training - Methodology of the Review36 END OF PART I

IAEA Basic IRRS Training - Methodology of the Review37 BASIC IRRS TRAINING MODULE 5 – PART II

IAEA Detailed contents part II Data collection methods 1.Reviewing additional written material (1-3) 2.Taking notes (1-3) 3.Interviews (1-5) 4.Good interviewing practices (1-8) 5.Direct observations (1-4) 6.Site visits (1-8) Basic IRRS Training - Methodology of the Review38

IAEA ADDITIONAL WRITTEN MATERIAL Basic IRRS Training - Methodology of the Review39

IAEA Requesting and reviewing additional written material (1)  The first review of the ARM often indicates that additional documents are needed for the understanding of an issue  It is not always possible to get additional (translated) documents before the mission, but the IAEA Coordinator could check this possibility in important cases  During the mission, additional material in the form of regulatory body documents, presentations and examples of practical applications will need to be requested and reviewed in order to be able to analyse and formulate conclusions, recommendations, suggestions and good practices Basic IRRS Training - Methodology of the Review40

IAEA Requesting and reviewing additional written material (2)  Since no ARM is fully complete, examples of such additional documents are given in the IRRS Guidelines appendix V, typical examples are: Lower level Management System documents, such as procedures and work instructions Annual activity planning and follow up documents Training plans and programmes for the regulatory staff Inspection reports Event reports Safety analysis reports or parts of such reports MoU with other national bodies Basic IRRS Training - Methodology of the Review41

IAEA Requesting and reviewing additional written material (3) Basic IRRS Training - Methodology of the Review42  For documents reviewed during the mission, full translations can often not be provided, ways to deal with this are: Ask for a translated summary Ask the counterpart to explain the contents of the document Use the Google translator, if possible  Most regulatory documents are likely available on the regulatory body´s intranet, ask the counterpart to demonstrate how the documents are accessed and can be used by the staff

IAEA TAKING NOTES Basic IRRS Training - Methodology of the Review43

IAEA Taking notes (1)  IRRS reviewers will compile a considerable volume of observations on various subjects and it is essential to take notes of every observation at the earliest opportunity, preferably during interviews, site visits etc.  Memory will not be reliable  It may be necessary to go back to old notes after several days  Also take notes of findings which may be relevant for colleagues, be a team worker  Working in pairs makes accurate recording much easier Basic IRRS Training - Methodology of the Review44

IAEA Taking notes (2) Technical notes may include:  Official names or titles of organizational units or persons met  Summary of points recorded or actions observed and their sources  Comments on any regulatory technical or policy issue observations  Comments on the role, responsibilities, and effectiveness of the organization  Documentation obtained or reviewed  Comments on strengths and areas for improvement within the organization, as perceived at the time  A list of issues to be brought to the attention of other team members  The full meaning of abbreviations or acronyms used  Information needed to complete those parts of questionnaires not previously completed Basic IRRS Training - Methodology of the Review45

IAEA Taking notes (3)  The accuracy of the notes should be checked with the counterpart after the interview or observation  Before the daily team meeting the notes of the day should be reviewed, key issues extracted and conclusions made for presentation by the Review Group Leader at the daily team meeting  After the daily team meeting, the notes of the day should be used to build the draft mission report  All notes should be kept by the reviewers until the mission report has been issued Basic IRRS Training - Methodology of the Review46

IAEA INTERVIEWS Basic IRRS Training - Methodology of the Review47

IAEA Interviews (1) During an IRRS mission, interviews will typically be conducted with personnel from:  The regulatory body  Government departments performing regulatory functions or having responsibility for the regulatory body  Advisory committees  Technical support organizations  Research institutes  Regulated organizations (licensees) Basic IRRS Training - Methodology of the Review48

IAEA Interviews (2) to gather information not covered by the written material to seek clarification of written information provided provides an opportunity for host regulatory body staff to discuss their practices and professional opinions with the reviewers Basic IRRS Training - Methodology of the Review49

IAEA Interviews (3)  The basis for most interviews in an IRRS mission is the host country´s self-assessment and other information included in the ARM  IRRS Team members will verify the information provided and enquire further where necessary to ensure they have a full understanding  Reviewers may deviate from the structure and contents of the self-assessment, and where alternative approaches might be beneficial in resolving untypical, difficult or contentious issues  Particular strengths becoming apparent in the interviews should be highlighted in the mission report and if appropriate be identified as good practice Basic IRRS Training - Methodology of the Review50

IAEA Interviews (4) Interviews are typically used to:  Review initial open issues arising out of the documentation review  Identify arrangements, authorities, duties and responsibilities of the regulatory body  Compare these arrangements with the IAEA Safety Standards and determine potential differences, as well as make an informal judgement on the adequacy of the host regulatory body´s practices  Examine the relationship between the regulatory body, its stakeholders and all those bodies involved in the subject area  Identify additional regulatory technical and policy issues  Judge whether there is a need to propose a recommendation or suggestion in the topic area reviewed Basic IRRS Training - Methodology of the Review51

IAEA Interviews (5)  Interviews should be conducted in a good sprit as a mutual exchange of views and not as an interrogation  The reviewer should lead the interview but allow time for counterparts to explain and to contribute to the body of knowledge on the topic area  To combine the interview with a description or demonstration of examples of work carried out, such as a case study, is an effective way to illustrate specific points  Questions should be prepared in advance and asked in a logical sequence  A general question should be followed by appropriate supplementary questions in order to obtain sufficient evidence to support the original response Basic IRRS Training - Methodology of the Review52

IAEA Good interviewing practices (1) Prepare properly for each interview:  Decide review area to be covered  Consider which documents and activities should be reviewed prior to the interview  Develop several questions to cover the area  Formulate to avoid simple answers like ”yes” and ”no”  Develop further questions if necessary  Select the ”final” set of questions for the interview Basic IRRS Training - Methodology of the Review53

IAEA Good interviewing practices (2)  Ask questions only to persons who are familiar with the area under review  Listen carefully to get key information for further investigation  Ask for clarification and supporting documents where necessary  If unclear, ask another person for the same information to confirm response  Keep the right distance between you and the interviewee, consider cultural aspects Basic IRRS Training - Methodology of the Review54

IAEA Good interviewing practices (3)  Chose a proper place for the interview  Make sure your words are clear and easy to understand  Control your body language and try to interpret the body language of your interviewee  Maintain a professional attitude at all times  Don´t allow your own perceptions to get in the way of your duties towards others Basic IRRS Training - Methodology of the Review55

IAEA Good interviewing practices (4) At the very beginning  Let the interviewee calm down and show a positive attitude  Explain review targets*  Ask for a short presentation, if needed, just to explain main activities, roles and responsibilities in the reviewing area * For nervous interviewees, emphasise that you ask because you would like to know how the organization works, not in order to inspect or to assess individual performance Basic IRRS Training - Methodology of the Review56

IAEA Good interviewing practices (5) How to ask?  Keep questions simple and focussed  Use open ended questions  Use a funnel approach: broad leading to specific questions  Anticipate unsatisfactory replies: have a means to deal with them  Avoid jargon  Avoid devious or trick questions  Focus on facts, ask for confirmation Basic IRRS Training - Methodology of the Review57

IAEA Good interviewing practices (6) How to listen? (1/2)  Be interested - show enthusiasm Be within comfortable listening distance Do not look down on the person: sit or stand the same as the person you are interviewing  Maintain eye contact  Do not walk away while listening  Use the right body language: polite gestures are good  Listen to the answer before asking next question Basic IRRS Training - Methodology of the Review58

IAEA Good interviewing practices (7) How to listen? (2/2)  Do not let note taking interfere with your listening Take only brief notes while listening Add more detail as soon as possible from memory  If you don´t understand the answer, ask for clarification Do not wait until the next day Do not assume, ask for confirmation Discuss with counterparts  Request copies of documents for later study  Don´t assume, ask questions Basic IRRS Training - Methodology of the Review59

IAEA Good interviewing practices (8) If an interpreter is used during interviews, the translations will take time. Try to make it easy for the interpreter by talking slowly and clearly and in a few sentences at a time before allowing translation. If possible give the interpreter the list of questions beforehand. Basic IRRS Training - Methodology of the Review60

IAEA DIRECT OBSERVATIONS Basic IRRS Training - Methodology of the Review61

IAEA Direct observations (1) Opportunities to observe work activities should be taken both at the regulatory body offices and during site visits. Direct observation can provide:  Valuable information complementary to the review of written material and the interviews  Opportunity for personal contact between the regulatory staff and reviewers  Better understanding of existing regulatory and policy issues through concrete examples  Good basis for continued discussions and exchange of professional knowledge and experience Basic IRRS Training - Methodology of the Review62

IAEA Direct observations (2) Activities to observe could be:  Management meetings  Safety committee meetings  Meetings with licensees and other stakeholders  Project review meetings  Planning of inspections and other regulatory tasks  Management system audits  Training sessions  Emergency exercises Basic IRRS Training - Methodology of the Review63

IAEA Direct observations (3) From the observations one should gain an improved understanding of e.g.:  How regulatory and administrative procedures are applied in practice  Attitudes and commitment of the regulatory staff  Work effectiveness  The use of electronic documentation systems and other administrative aids  Traceability of regulatory activities and the decision making process Basic IRRS Training - Methodology of the Review64

IAEA Direct observations (4) Remember:  A precondition for using observation as a data source for conclusions is that the context of the observed activity is understood by the reviewer  Prepare by checking relevant procedures and if necessary ask the counterpart to explain the significance of the activity and to translate (discretely) during the activity  Be aware that presence of an observer could affect the observed activity  Observations should be properly recorded in technical notes and reflected in the mission report. Basic IRRS Training - Methodology of the Review65

IAEA SITE VISITS Basic IRRS Training - Methodology of the Review66

IAEA Site visits (1), purpose Certain interviews and direct observations of regulatory practices and activities (typically inspections) should be carried out on the sites of authorized organizations Such visits will enable a better understanding of how the regulatory body discharges its responsibility and give an opportunity for reviewers to meet with licensees Basic IRRS Training - Methodology of the Review67

IAEA Site visits (2), facilities of interest To minimize travel time and reduce disruption for the visited facilities, maximum one visit is typically arranged to each broad category of facility, such as  Nuclear power plant  Research reactor  Fuel cycle facility  Waste management facility  Hospital  Industry using radioactive source  Environmental monitoring laboratory Basic IRRS Training - Methodology of the Review68

IAEA Site visits (3), types of information Through observations of work activities and interviews with the regulatory body staff and the licensee staff types of information to be gathered may include:  Perceived and actual roles and responsibilities of the regulator at the visited site  Resources to fulfil those responsibilities, including facilities, equipment and staffing  Knowledge, skills and abilities of the regulatory inspectors to perform inspections  Effectiveness of the regulatory processes  Elements relevant to policy issues  Relations between the regulatory body and the licensee Basic IRRS Training - Methodology of the Review69

IAEA Site visits (4), planning  Site visits need to be scheduled early in the mission in order to give time to incorporate the observations in the mission report  At least two IRRS reviewers should be assigned to visit a major facility such as a NPP  Counterparts need to notify facilities well in advance of the intended visit by IRRS reviewers and inform about the purpose with the visit  Prior to the visit IRRS reviewers need to prepare by obtaining an understanding of the role and function of the facility and identify topics and issues already raised and relevant for the IRRS mission Basic IRRS Training - Methodology of the Review70

IAEA Site visits (5), conduct  The visit should start with an opening statement to the site host informing about the scope of the IRRS mission, the purpose of the visit and questions to be asked  It is important that staff of the visited facility understand in advance that the visit is not an IAEA review of their facility but is conducted solely to observe practices and actions of the regulatory staff  Reviewers should be prepared to accept changes in the arrangements made by the visited facility while making every effort to cover all the topics on their agenda Basic IRRS Training - Methodology of the Review71

IAEA Site visits (6) conduct  IRRS reviewers should be accompanied by a host country counterpart from the head office to facilitate visit logistics  On site the IRRS reviewers should concentrate on observing the site inspector conducting a normal inspection or other routine regulatory tasks  Before the walk around, the inspector should brief the reviewers about the planning process for the inspection, if this has not been done before the visit  During the walk around the reviewers should act as observers and not interfere in the inspection activities Basic IRRS Training - Methodology of the Review72

IAEA Site visits (7) conduct  Reviewers may ask the inspector to explain what he is doing if this does not disturb his/her activities  After the inspection, the inspector should inform the reviewers about his/her observations and conclusions and what the next steps are  If possible before leaving the site, the reviewers should also interview the senior manager at the visited facility to discuss the nature of the relationship with the regulatory body  These questions could deal with communications arrangements and how the regulatory body is perceived with regard to resources, competence, working methods and formalities such as regulations and enforcement practices Basic IRRS Training - Methodology of the Review73

IAEA Site visits (8) conduct  Such information will provide a valuable complement to the information provided by the regulatory body itself  During interviews with licensees, the preference is that host regulatory body staff is not present, in order to facilitate open and frank discussions  After the visit, reviewers should discuss their findings with the full IRRS team at the next Daily Team Meeting Basic IRRS Training - Methodology of the Review74

IAEA Basic IRRS Training - Methodology of the Review75 END OF PART II

IAEA Summary (1) An IRRS is not possible without good team work Every reviewer needs to prepare well and be efficient, professional and skilled Reviewers are assigned to review groups (A-F) Good team workers are flexible and open minded The daily team meeting is the most important instrument for discussing conclusions and agreeing recommendations, suggestions and good practices Basic IRRS Training - Methodology of the Review76

IAEA Summary (2) To collect data during the mission: Additional documentation needs to be reviewed Copious notes should be taken at the earliest opportunity Lists of questions should be prepared for interviews Interviews should be conducted in a friendly and professional manner and encourage the counterpart to provide substantial information, don´t assume – ask Direct observation of work activities could provide valuable additional information Site visits are vital for understanding of the regulatory effectiveness and the relations with the licensees Basic IRRS Training - Methodology of the Review77

IAEA THANK YOU FOR YOUR ATTENTION Basic IRRS Training - Methodology of the Review78 Photo: V. Friedrich This activity is conducted by the IAEA, with funding by the European Union. The views expressed in this presentation do not necessarily reflect the views of the European Commission

IAEA Issues to discuss 1. When assessing the practices of the host regulatory body, to what extent is it proper to use your own organization´s practices as a reference? 2. How to act if you suspect that an interview person is not giving you an objective picture? 3. How to act if your counterpart strongly disagrees with a proposed recommendation? Basic IRRS Training - Methodology of the Review79