1. 1 Auditing Your Fusion Center Privacy Policy

2. 22 Recommendations to the program resulting in improvements Updates to privacy documentation Informal discussions about lessons learned Formal report either internal or publicly available Heightened awareness by all participants about privacy Early issue identification and remediation What Are the Outcomes and Benefits of a Privacy Audit?

3. 33 The privacy policy is a series of promises to demonstrate the enter is performing This means BOTH: – Reviewing the policy for accuracy and keeping it up to date (“type A”) – Auditing center use of data per privacy policy (“type B”) Privacy Policies from an audit perspective Privacy Policy Development Implementation Audit

4. 44 Annual review of policy for accuracy – Are governance structures in place (e.g., privacy/CL committee, audit committee)? – Are standard operating procedures and other policies referenced in place? – Are access and dissemination logs maintained? Annual review of policy for currency – Do changes need to be made in response to changes in applicable law, technology, the purpose and use of the information systems, and public expectations? Resource: Compliance Verification Tool Type A – Audit Keeping Policy Accurate and Current

5. 55 Goes a step beyond verifying that policies and procedures are in place Requires development of review mechanisms that demonstrate that the privacy policy and policies and procedures are adhered to Looks at center’s actual use of data – Data quality (e.g., labeling/tagging) – Data use (e.g., access and dissemination logs) – Retention (e.g., Are records maintained within the specified retention schedule?) Type B – Auditing Center Use of Data

6. 66 May be required in existing privacy policy Opportunity to identify strengths, weaknesses, and corrective actions Heightens awareness and the importance of adherence to the policy Demonstrates center accountability and sends a message to the public about your commitment to adhere to the policy Ultimately strengthens the program! The value of audits

7. 77 Auditing Fundamentals: Elements of a Finding Criteria, Condition, Cause, and Effect Criteria: The laws, regulations, contracts, grant agreements, standards, measures, expected performance, defined business practices, and benchmarks against which performance is compared or evaluated. Criteria identify the required or desired state or expectation with respect to the program or operation. Criteria provide a context for evaluating evidence and understanding the findings. —Generally Accepted Government Auditing Standards (GAGAS) 4.11 The fusion center’s privacy policy is the criteria that the center and other oversight entities will use to design an audit.

8. 88 Fusion Center Privacy Policy as Criteria (Example 1: Data Quality) Deconstruct the privacy policy into a series of requirements against which to review center performance. Example 1: The P/CRCL Officer or designee audits the quality of information received from an originating agency to ensure, to the extent possible, that it: (1) is accurate and complete, (2) does not include incorrectly merged information, (3) is not out of date, (4) can be verified, (5) does not lack adequate context such that the rights of the individual may be affected, and (6) was not gathered in violation of federal, state, or local laws or ordinances. Audit Procedure: Identify applicable systems and determine whether information obtained from an originating agency is appropriately tagged with the source, and review selected records from the system against the six items identified in the privacy/CL policy.

9. 99 Fusion Center Privacy Policy as Criteria (Example 2: Data Use) The fusion center will maintain an access log and dissemination record (audit trail) when the database is accessed or information is disseminated from the intelligence system. This dissemination record contains the following information: the date of dissemination of the information, the name of the individual requesting the information, the name of the agency requesting the information, the reason for the release of the information, the description of the information provided to the requestor, and the name of the fusion center person disseminating the information. Audit Procedure: Obtain and review access logs from intelligence systems and verify that the log is designed to capture the information specified in the privacy policy/CL and that information is properly filled out. Design considerations: Number of systems to be reviewed? Number of records to review? It may not be feasible to review every record or log entry or to even review a statistically significant sample.

10. 10 Auditing Fundamentals: Elements of a Finding —Condition Criteria, Condition, Cause, and Effect Condition: Condition is a situation that exists. The condition is determined and documented during the audit. —GAGAS 4.12 Possible Conditions for Example 1 (Data Quality Audit): I.A review of 30 percent of records from 3 of the center’s largest information systems found that most records specified 4 of the 6 data quality elements in the center’s policy. II.We could not assess whether records originating from another agency met the 6 criteria. Possible Conditions for Example 2 (Data Use Audit): I.A review of the audit log for 24 randomly selected days in 2011 (2 days per month) found that the log was designed to capture all of the information specified in the privacy policy. The logs we reviewed demonstrated that all fields were filled out. II.We found that the audit log was designed to capture only 3 of the 6 data elements specified in the privacy policy.

11. 11 Elements of a Finding—Cause Criteria, Condition, Cause, and Effect Cause: The cause identifies the reason or explanation for the condition or the factors responsible for the difference between the situation that exists (condition) and the required or desired state (criteria), which may also serve as a basis for recommendations for corrective actions. Common factors include poorly designed policies, procedures, or criteria; inconsistent, incomplete, or incorrect implementation; or factors beyond the control of program management. Auditors may assess whether the evidence provides a reasonable and convincing argument for why the stated cause is the key factor or factors contributing to the difference between the condition and the criteria. —GAGAS 4.13 Potential Cause(s) for Examples 1 and 2: Systems were not designed to capture the information specified in the policy Privacy/CL policy was not implemented—analysts were not aware of their responsibilities to tag the information Not feasible to capture certain aspects of this information in the log

12. 12 Elements of a Finding – Effect or Potential Effect Effect or potential effect: The effect is a clear, logical link to establish the impact or potential impact of the difference between the situation that exists (condition) and the required or desired state (criteria). The effect or potential effect identifies the outcomes or consequences of the condition. When the audit objectives include identifying the actual or potential consequences of a condition that varies (either positively or negatively) from the criteria identified in the audit, “effect” is a measure of those consequences. Effect or potential effect may be used to demonstrate the need for corrective action in response to identified problems or relevant risks. —GAGAS 4.14 In privacy audits, look for a direct effect on an individual or a potential effect. Often this could be the potential for action to be taken against an individual based on inaccurate or untimely information.

13. 13 Audit Recommendations Should logically flow from findings. If a deficiency is found, the recommendation should be targeted at correcting the deficiency Example recommendations: – The center should modify its systems to ensure that appropriate databases capture fields as specified in the privacy/civil liberties policy – Standard operating procedures and training should be put in place to inform analysts of their responsibilities to record information related to sharing records with external entities Document your audit findings and recommendations Follow up on your recommendations, whether made by the P/CRCL Officer or an oversight entity

14. 14 Recommended Areas for Audits Data quality Data use and sharing—review your logs! Contributions to the Nationwide Suspicious Activity Reporting Initiative (NSI) – Use the ISE-SAR Functional Standard (FS) Version 1.5 as your criteria – Review the content of NSI submissions for adherence to the FS – Remove reports that do not meet the FS or require updates that make NSI submissions FS-compliant

15. 15 Fusion center P/CRCL Officer-led State auditing agency State Inspector General Third-party audit Peer-to- Peer audit (DHS is a resource for matching fusion centers) Possible Resources for Fusion Center Audits