1. External presentation The views and opinions expressed by guest speakers do not necessarily reflect the views or position of the Clean Energy Regulator. Heather Watson, Chartered Accountants Australia and New Zealand

2. Quality Control PROMOTING CONFIDENCE IN JUDGEMENTS

3. Clean Energy Regulator Annual Scheme Audit and Assurance Workshop HEATHER WATSON FCA 31 MARCH 2015

4. Assurance and auditing – which is it? ART?SCIENCE?

5. What is an art? a superior skill that you can learn by study, practice and observation a habit of thinking, doing, or making that demonstrates systematic discipline based on principles arts are based on principles, whether the practitioner know them or not. An art is not just a series or procedures or methods. There can be many methods inside an art. Art gives strategic purpose to methods 5

6. What is a science?  systematically acquired knowledge that is verifiable  any systematic knowledge-base or prescriptive practice that is capable of resulting in a correct prediction, or reliably-predictable type of outcome  a particular discipline or branch of learning, especially one dealing with measurable or systematic principles rather than intuition or natural ability 6

7. Profession al Judgement in ASAE 3000 The assurance practitioner shall exercise professional judgement in planning and performing an assurance engagement, including determining the nature, timing and extent of procedures. Paragraph 38

8. Profession al Judgement in ASAE 3000 Professional judgement is essential to the proper conduct of an assurance engagement. This is because interpretation of relevant ethical requirements and relevant ASAEs and the informed decisions required throughout the engagement cannot be made without the application of relevant training, knowledge, and experience to the facts and circumstances. Paragraph A81

9. Profession al Judgement in ASAE 3000 Professional judgement is necessary in particular regarding decisions about:  Materiality and engagement risk  The nature, timing, and extent of procedures used to meet the requirements of relevant ASAEs and obtain evidence  Evaluating whether sufficient appropriate evidence has been obtained, and whether more needs to be done to achieve the objectives of this ASAE and any relevant subject matter specific ASAE. In particular, in the case of a limited assurance engagement, professional judgement is required in evaluating whether a meaningful level of assurance has been obtained  The appropriate conclusions to draw based on the evidence obtained. Paragraph A81

10. Profession al Judgement in ASAE 3000 The distinguishing feature of the professional judgement expected of an assurance practitioner is that it is exercised by an assurance practitioner whose training, knowledge and experience have assisted in developing the necessary competencies to achieve reasonable judgements. Paragraph A82

11. Profession al Judgement in ASAE 3000 The exercise of professional judgement in any particular case is based on the facts and circumstances that are known by the assurance practitioner. Paragraph A83

12. Profession al Judgement in ASAE 3000 Consultation on difficult or contentious matters during the course of the engagement, both within the engagement team and between the engagement team and others at the appropriate level within or outside the firm assist the assurance practitioner in making informed and reasonable judgements, including the extent to which particular items in the subject matter information are affected by judgement of the appropriate party. Paragraph A83

13. Profession al Judgement in ASAE 3000 Professional judgement can be evaluated based on whether the judgement reached reflects a competent application of assurance and measurement or evaluation principles and is appropriate in the light of, and consistent with, the facts and circumstances that were known to the assurance practitioner up to the date of the assurance practitioner’s assurance report. Paragraph A84

14. Profession al Judgement in ASAE 3000 Professional judgement needs to be exercised throughout the engagement. It also needs to be appropriately documented. In this regard, paragraph 79 requires the assurance practitioner to prepare documentation sufficient to enable an experienced assurance practitioner, having no previous connection with the engagement, to understand the significant professional judgements made in reaching conclusions on significant matters arising during the engagement. Professional judgement is not to be used as the justification for decisions that are not otherwise supported by the facts and circumstances of the engagement or sufficient appropriate evidence. Paragraph A85

15. Apply professional judgement when:  determining the extent of the understanding of subject matter and engagement circumstances required  planning and performing the engagement  evaluating the quantity and quality of evidence  assessing the extent of documentation required.

16. ICAA thought leadership paper on professional judgement, 2006  ‘judgment is the cornerstone of auditing’  ‘judgment is the most important factor in the making of an audit’ (AICPA 1955)  ‘judgment must inevitably play a major role in auditing’ (Mautz 1959) 16

17. ICAA thought leadership paper on professional judgement, 2006 ‘Regulators around the world seem to be taking the attitude that any error in judgement is unacceptable and that the public have the right to a zero tolerance on errors in judgement by auditors or the existence of financial fraud.’

19. We all make mistakes “Of course experts make mistakes … But precisely because they are experts, they are more likely to be right than ordinary people. Brain surgeons make mistakes, but they know more than the rest of us about brain surgery; lawyers make mistakes, but they know more than most people about the law.” Sunstein (2002), p77, quoted in ICAA Thought Leadership Paper

20. Professional Judgement – Decision Making What sort of intuitive choice processes do you use to manipulate substantial knowledge to arrive at a decision? Intuitive judgements are made using information that has been processed and transformed by the human mind.

21. Cognitive Limitations

22. Limited Information Processing Capacity  perception of information is not comprehensive but selective – people only see what they want to see  processing is mainly done in a sequential manner – and the actual sequence in which information is processed may bias their judgement  people do not possess intuitive ‘calculators’ - they can’t perform optimal calculations and use simple procedures, rules or tricks instead  people have limited memory capacity – memory is formed by reconstructing fragments of information Hogarth, Robin

23. Judgement Traps Habits/Rules of thumb – it worked in the past Anchoring and adjustment – is the anchor suitable? Representativeness – likelihood based on similarity Excessive confidence – experts don’t doubt themselves as much as novices Unrealistic confidence – based on incomplete information Emotional stress – impairs judgement Preferences of others – undue influence Own preference – wishful thinking affects assessments Hogarth, Robin

24. How do we overcome our imperfections?

25. Firm wide quality control

26. ASQC1 definition of quality control Engagement quality control review means a process designed to provide an objective evaluation, on or before the date of the report, of the significant judgements the engagement team made and the conclusions it reached in formulating the report.

27. Quality Control Elements  Leadership responsibilities for quality within the firm  Relevant ethical requirements  Acceptance and continuance of client relationships and specific engagements  Human resources  Engagement performance  Monitoring ASQC1.16 27

28. Human Resources Perform engagements in accordance with AUASB Standards, relevant ethical requirements, and applicable legal and regulatory requirements; and Enable the firm or engagement partners to issue reports that are appropriate in the circumstances. ASQC1.31 28 The firm shall…establish policies and procedures to assign appropriate personnel with the necessary competence, and capabilities to:

29. Assigning personnel  understanding of, and practical experience with, engagements of a similar nature and complexity through appropriate training and participation  understanding of AUASB Standards, relevant ethical requirements, and applicable legal and regulatory requirements  technical knowledge and expertise, including knowledge of relevant information technology  knowledge of relevant industries in which the clients operate  ability to apply professional judgement  understanding of the firm's quality control policies and procedures. ASQC1.A31 29 The firm's assignment of engagement teams and the determination of the level of supervision required, include for example, consideration of the engagement team's:

31. Consultation The firm shall establish policies and procedures designed to provide it with reasonable assurance that these items are documented:  appropriate consultation takes place on difficult or contentious matters  sufficient resources are available to enable appropriate consultation to take place  the nature and scope of consultation, including who was consulted  conclusions resulting from consultation, which are agreed And that conclusions resulting from consultations are implemented or the reasons alternative courses of action from consultations were undertaken. ASQC1.34

33. How to Consult – ASQC1 Consultation includes discussion at the appropriate professional level, with individuals within or outside the firm who have specialised expertise. Consultation uses appropriate research resources as well as the collective experience and technical expertise of the firm. Consultation helps promote quality and improves the application of professional judgement. Appropriate recognition of consultation in the firm’s policies and procedures helps promote a culture in which consultation is recognised as a strength and encourages personnel to consult on difficult or contentious matters.

34. How to Consult – ASQC1 Effective consultation on significant technical, ethical and other matters within the firm, or where applicable, outside the firm can be achieved when those consulted:  are given all the relevant facts that will enable them to provide informed advice  have appropriate knowledge, seniority and experience And when conclusions resulting from consultations are appropriately documented and implemented.

35. How to Consult – ASQC1 Documentation of consultations with other professionals that involve difficult or contentious matters that is sufficiently complete and detailed contributes to an understanding of the:  issue on which consultation was sought  results of the consultation, including any decisions taken, the basis for those decisions and how they were implemented.

36. How to Consult – ASQC1 A firm needing to consult externally, for example, a firm without appropriate internal resources, may take advantage of advisory services provided by:  other firms  professional and regulatory bodies  commercial organisations that provide relevant quality control services. Before contracting for such services, consideration of the competence and capabilities of the external provider helps the firm to determine whether the external provider is suitably qualified for that purpose. Small firms

37. Documenting Judgements

38. Documentation of judgements The auditor shall prepare audit documentation that is sufficient to enable an experienced auditor, having no previous connection with the audit, to understand... Significant matters arising during the audit, the conclusions reached thereon, and significant professional judgements made in reaching those conclusions. ASA230.8

39. more judgement = more documentation less judgement = less documentation

40. Documentation of judgements An important factor in determining the form, content and extent of audit documentation of significant matters is the extent of professional judgement exercised in performing the work and evaluating the results. ASA230.A9

41. JudgementDocumentation Quality Judgement

42. Documentation of judgements – relationship with quality Documentation of the professional judgements made, where significant, serves to explain the auditor's conclusions and to reinforce the quality of the judgement. ASA230.A9

43. Documentation of judgement processes Ensure working papers document:  consultation processes  knowledge and understanding of circumstances that were considered when arriving at judgements  consideration of options not selected.

44. Determination of materiality Selection and scope of procedures Significant matters Responses to risks Application of materiality Extent of documentation Evaluating management’s judgements Inconsistent evidence Compliance with ethical requirements Persuasiveness of evidence Identification of risks of material misstatement Forming the opinion

45. Specific judgements to document  determination and application of materiality  compliance with ethical requirements  the selection and scope of appropriate audit procedures  the assessment and evaluation of available options and results, including the persuasiveness of audit evidence  the extent of documentation of audit plans, procedures, results, conclusions and communications. ASA100.37

46. Documentation of judgements  identification of risks of material misstatement  designing and implementing appropriate responses to the assessed risks  forming the opinion ASA200.7

47. Documentation of judgements  evaluation of management's judgements in applying the entity's applicable financial reporting framework.  drawing of conclusions based on the audit evidence obtained, for example, assessing the reasonableness of the estimates made by management in preparing the financial report ASA200.A23

48. Documentation of judgements in respect of significant matters  matters that give rise to significant risks  results of audit procedures indicating that the financial report could be materially misstated  results of audit procedures indicating a need to revise the auditor’s previous assessments of the risks of material misstatement and the auditor’s response to those risks  situations where applying standards difficult  findings that could result in modifications to opinion or report ASA230.A8

49. ASA 230 examples The rationale for the auditor's conclusion when a requirement provides that the auditor ‘shall consider’ certain information or factors, and that consideration is significant in the context of the particular engagement. The basis for the auditor's conclusion on the reasonableness of areas of subjective judgements (for example, the reasonableness of significant accounting estimates). The basis for the auditor's conclusions about the authenticity of a document when further investigation (such as making appropriate use of an expert or of confirmation procedures) is undertaken in response to conditions identified during the audit that caused the auditor to believe that the document may not be authentic.” (ASA 230.A11) ASA230.A8

50. Documentation for ‘Ordinarily’ Ordinarily – means the explanatory guidance indicates practical methods or means by which mandatory requirements may be complied with and is to be read in the following context: where the word ‘ordinarily’ is used, the auditor exercises professional judgement in considering:  whether the noted circumstances apply to the current audit, review or other assurance engagement; and  if so, whether the suggested procedures are appropriate to perform;  or where there are alternative procedures which are more appropriate, whether these alternative procedures are to be performed. AUASB Glossary 50

51. Summary of judgements The auditor may consider it helpful to prepare and retain as part of the audit documentation a summary (sometimes known as a completion memorandum) that describes the significant matters identified during the audit and how they were addressed, or that includes cross- references to other relevant supporting audit documentation that provides such information....the preparation of such a summary may assist the auditor's consideration of the significant matters. (ASA 230.A11) ASA230.A8

52. How good is your judgemen t? As good as the documentati on supporting it.

53. Assurance skill is reflected in the documentation:  Apply the standards as would an artist  Document your application as would a scientist 53

54. References Hogarth, Robin, 1980, Judgement and Choice, John Wiley and Sons Trotman, K.T. Professional judgment: are auditors being held to a higher standard than other professionals? Institute of Chartered Accountants in Australia. 2006 Auditor scepticism: raising the bar, Auditing Practices Board of the Financial Reporting Council, United Kingdom. 2010 Carpenter, TD and Reimers, JL, 2009 Professional Scepticism: The Effects of a Partner’s Influence and the Presence of Fraud on Auditors’ Fraud Judgments and Actions