1 PUG Challenge Americas 2014 Click to edit Master title style PUG Challenge EMEA 2014 – Dusseldorf, Germany Tales from the Audit Trails Presented by: Mike Furgal

2 PUG Challenge Americas 2014 Introductions Mike Furgal –Progress employee from 1989 Short time at Bravepoint from 2012 until 2014 –Progress OpenEdge Database Expert

3 PUG Challenge Americas 2014 Introduction - BravePoint Managed Database Services Databases 50+ TB in DB space 75,000+ connected users Pro2 Replication Real Time Replication SQL target 600+ Deployments

4 PUG Challenge Americas 2014 Disclaimer The techniques covered in this presentation are directed at the users and administrators of OE applications. Out of scope topics include: –Establishing Trusted User Identity –Adding Auditing Events to an application

5 PUG Challenge Americas 2014 Why Auditing? Regulatory requirements… –SEC (Sarbanes Oxley) –FDA (CFR 21 Part 11) –HIPAA –Immigration (I-9s) –And more Security Peace of mind

6 PUG Challenge Americas 2014 Auditing Options “Roll your own” ABL based solutions OpenEdge Auditing Let’s do a Poll: –Who has an audit system? –Who’s audit system is written in the ABL? –Who’s already using OE Auditing? –Who wishes they had auditing on their OE application?

7 PUG Challenge Americas 2014 “Roll your own” Solutions –Typically use Replication or Database Triggers –Pros Program in the ABL Very fine control –Cons May require access to application source code Defeatable/insecure High overhead Complications (SQL access for example.) Triggers are client based

8 PUG Challenge Americas 2014 OpenEdge Auditing Integrated in the product since 10.1A –No additional $ cost Built into the Database Engine –Low overhead –Supports both ABL and SQL transactions Secure and tamper proof Tools for archiving and reporting Unimpacted by ABL code constructs –ie: DISABLE TRIGGERS

9 PUG Challenge Americas 2014 OpenEdge Auditing Comprehensive Auditing Solution –Table based Auditing –Field Based Auditing –Allows capture of Before and After versions of the data Auditing includes Security so there is no tampering with data Some reporting capabilities built into the product

10 PUG Challenge Americas 2014 What does OE Auditing track? Changes to... – The Database Schema – The Application Data – Application Defined Events – The Security (New users, deleted users) – Database Encryption – The Audit Policies

11 PUG Challenge Americas 2014 Where is the data stored? Auditing adds tables to your production DB –Audit trail data is stored in: _aud-audit-data _aud-audit-data-value –Audit policy and controls are stored in: _aud-audit-policy _aud-event _aud-event-policy _aud-field-policy _aud-file-policy

12 PUG Challenge Americas 2014 What does basic Audit Data look like? _Audit-data-guid |"/unygEGMpaXiEXSzdJPDfQ" _Database-connection-id |"/unygEGMpaXiEXSzOlqlcw" _Client-session-uuid |"" _User-id |"root" _Audit-date-time | T18:06: :00 _Audit-event-group |"" _Db-guid |"4f5kvLayZrXiEXKzHE+akA" _Transaction-id |403 _Transaction-sequence |0 _Event-id |5101 _Event-context |"PUB.Customer^F84" _Application-context-id |"" _Event-detail |"Credit-Limit^F5^F12345^F22222" _Audit-custom-detail |"" _Audit-data-security-level |0 _Data-seal |""

13 PUG Challenge Americas 2014 Cust-Num^F4^F71^F^GName^F1^Fpocket billiards co.^F^GAddress^F1^F44 Saunders Ave.^F^GAddress2^F1^F^F^GCity^F1^FPhelan^F^GState^ F1^Fca^F^GCountry^F1^FUSA^F^GPhone^F1^F(818) ^F^GContact^F1^FLeon Aida^F^GSales- Rep^F1^FKIK^F^GComments^F1^F^F^GCredit- Limit^F5^F5000^F^GBalance^F5^F0^F^GTerms^F1^FNet3 0^F^GDiscount^F4^F50^F^GPostal-Code^F1^F92371^F" What is in the _Event-detail field? It can be a little… Or a lot…. "Credit-Limit^F5^F12345^F22222"

14 PUG Challenge Americas 2014 Enable Audit on the Database Enable auditing (with indexes inactive!) Shutdown and backup database Add storage areas for audit data and indexes

15 PUG Challenge Americas 2014 Define Audit Policies Use Audit Policy Maintenance Tool in GUI OpenEdge

16 PUG Challenge Americas 2014 Determining Auditing Requirements What type of things do you want to audit? How long do you keep your audit data? What performance impact can you tolerate? How does auditing fit into your disaster recovery plan? Who can access or manage your audit rules and data? What type of reporting/inquiry is required?

17 PUG Challenge Americas 2014 Roles and Responsibilities The auditing system requires an administrator –Best practice for this position has it as separate person from the DBA The administrator designates who can: –Administer the audit rules –Report on audit data –Archive or delete audit data Requires use of OE security

18 PUG Challenge Americas 2014 The Audit Data The audit tables have 27 indexes defined –Best practice is to have most turned off in production –We’ll see why in a moment Effective reporting requires that these indexes be active. Having the audit data in an archive database allows for reporting without negatively impacting production OpenEdge provides tools to securely move data into the archive database.

19 PUG Challenge Americas 2014 Overhead of Auditing Customer example: QAD System –Real world audit policies –Inventory Load ( &.2) Tested 5 scenarios: –Baseline without auditing –Auditing w/indexes inactive –Auditing w/indexes active –Auditing w/indexes inactive using value table –Auditing w/indexes active using value table

20 PUG Challenge Americas 2014 BI Logging

21 PUG Challenge Americas 2014 Record Creates & Locks

22 PUG Challenge Americas 2014 Overhead of Auditing on DB Size Keeping audit history in production may not seem significant Audit Data

23 PUG Challenge Americas 2014 Overhead of Auditing on space As the database grows Audit data takes up an immense amount of space Archiving out audit data keeps the database at an appropriate size - This will generate a lot of BI/AI activity

24 PUG Challenge Americas 2014 Archive Commands To periodically archive data out of production ready to load into the archive database To load the exported audit data into the archive database

25 PUG Challenge Americas 2014 Caveat It is not possible to truncate an area that contains Audit data. OpenEdge 11.2 proutil -C auditreconfig [ tablearea ] [ indexarea ] Recreates a new Audit Areaand moves all audit data to the new area

26 PUG Challenge Americas 2014 Reporting from Audit Data Things to consider: –Do you report off production? –Do you report off archive –Do you report from both? Timeliness of the incident being analyzed –Can alerts be set up? –This is a “roll your own” activity

27 PUG Challenge Americas 2014 OE Reporting Options

28 PUG Challenge Americas 2014 DEMO Start with a Sports database Enable Auditing Use Audit Policy Maintenance Tool –Audit customer table –Audit changes to customer max-credit Create a workload on db Polling process watching for changes to max-credit in _aud-audit-data Change the max credit and see what happens.

29 PUG Challenge Americas 2014 DEMO

30 PUG Challenge Americas 2014 Caveats Make sure you use deactivateidx on the production database audit tables –Spelling counts!!!! –If you rebuild all indexes all 27 audit indexes become active!!! Index rebuild causes worse performance You need to retune both BI and AI –Think about OE Replication and AI size impacts Don’t forget your audit policies during a dump/load! Managing 2 databases (production and archive) –Backups –Space management –Recovery Plans

31 PUG Challenge Americas 2014 Conclusions OE Auditing is a powerful tool Knowing what needs to be audited may not be obvious Use an iterative approach –Audit less than you need to start –Ramp as needed Make sure you have a solid data management plan Report and alert as needed

32 PUG Challenge Americas 2014 Thank You! Questions?