1. © 2015 IBM Corporation MasterCard and QMF: Providing Analytics for More Than DB2; DDZ-1281 Tom Glaser (MasterCard)

2. Installation of the Workstation Client QMF W/ DB2 Weekly Zero Counter Report Weekly Top 20 Report Access to Performance Database W/ solidDB W/ PostgreSQL W/ Oracle Moving data between different relational RDBMSs Joining tables from different RDBMSs QMF for Web Client (Job Scheduler Monthly Jobs) Workspaces Dashboard; DB2 zParms Using QMF for the IBM DB2 Analytics Accelerator Stored Procedures Hidden GEMS Agenda

3. Who is MasterCard? Consumers want better ways to pay. We invent them. Checkout lines are too slow. We help them move faster. Commuters are busy. We speed them on their way. Procurement is complicated. We make it simple. People want financial access. We find ways to serve them. Our Role

4. Who is MasterCard? Key Statistics 210 countries and territories 10,300 employees 43 billion processed transactions US $4.5 trillion* gross dollar volume * Excludes volume generated by Maestro and Cirrus cards All numbers as reported as of December 31, 2014 US $9.5 billion net revenue

5. 55 DB2 z/OS Subsystems Data Sharing and non Data Sharing V10 QMF for TSO, Workstation, Web Clients Relational DBMSs: DB2 Oracle SQL Server PostgreSQL solidDB MySQL MasterCard: Our Environment

6. Where does MasterCard use QMF? Web Client Workstation Windows TSO

7. SolidDB Oracle PostgreSQL IBM DB2 for z/OS QMF Web client Workstation client TSO client CDC Where does MasterCard use QMF?

8. Weekly Performance Reports: Zero Counter Report QMF TSO Client D2P1 METRIC D2P1 CB D2P2 D2P3 D2P4 ------------------------------------------------------------------------ BUFFERPOOL: - - - - - 4KUsed, Not 32K - - - - - 32KUsed, Not 4K - - - - - DM Crit Thresh - - - - - Expansion/Full - - - - - Expansion/GETM - - - - - GETPAGE/NoBuff 1511306 1481379 - - 910025 Merge/No Buffer - - - - - WrkFile/BufRes - - - - - WrkFile/No Buf 2365519 - - - - DATA SHARING: - - - - - CF Lack of Stor - - - - - EngNotAvaiPLock - - - - - ExplicCrosInval - - - - - GBP CastoutThrs - - - - - MVX XES ResCont - - - - - EDM POOL: - - - - - EDM DBD Full - - - - -

9. Weekly Performance Reports: Top 20 Report #2:TOP CPU CONSUMERS BY PKGS FOR ALL DB2 SUBSYSTEMS MASTERCARD TOTAL DB2 DB2 CPU CPU DB2 PACKAGE HOURS COST SSID LPAR -------- ---------- ---------- ---- ---- SYSLH200 296.08 19244.93 DDP6 CPUI DSNTIAUL 275.44 17903.82 DDP6 CPUI CZIUOWPG 197.71 12851.07 DDP6 CPUI DSNTIAUL 197.00 12804.89 D2P1 CPUA GP038750 171.12 11122.66 D2P1 CPUA DSNTIAUL 162.90 10588.35 DBP6 CPUJ DSNTIAUL 139.82 9088.56 D2P1 CPUC SYSSH200 133.65 8686.97 D2P1 CPUC DA831010 131.53 8549.55 DDP6 CPUI SYSSH200 129.48 8415.90 DBP7 CPUJ SYSSH200 114.12 7418.10 DAP7 CPUI SYSSH200 102.99 6694.20 D2P1 CPUA QMF TSO Client…one more example

10. 08/07/2015 - 08/08/2015 OBJECT ELAPSED SYNC SYNC READS SYNC READS CORRID NAME BP GPs HRS READS PCT CUM PCT ------ ------------------ ------ -------------- -------- ---------------- ---------- ---------- ENTRPS DCTGN_XCHG_GENRFIN BP34 7,205,738 2.77 2,735,878 51 51 DCIGN1 BP33 15,010,822 1.31 2,665,291 49 100 ---------------- ---------- ---------- * 5,401,169 2 2 ENTRXH ADDRESS_TB BP32 36,729,225 5.78 19,707,071 72 72 ADFA21X1 BP31 105,670,833 1.57 7,717,667 28 100 ---------------- ---------- ---------- * 27,424,738 11 14 QMF Workstation Client….adhoc / perform DB F O R M S …Cumulative Percentage …Percentage

11. Microsoft Software Center: - We test to make sure DB2 Connect has been installed (for the *.jar files) Installation for the QMF Workstation Client

12. Two Repositories: - Used by QMF for the Workstation and Web clients - Both Repositories are identical Installation for QMF for Workstation Client…cont Going with a shared repository allows us to add one ssid and everyone has access to it. Simple! xxxxxxxxx

13. solidDB In-memory Database: CDC Replication from DB2 to solidDB Command line interface on AIX We needed a better method of querying solidDB so we turned to the Workstation and Web clients of QMF.

14. Adding solidDB Driver: solidDB…cont

15. AIX server + Environment solidDB…cont

16. Which do you prefer: OR solidDB…cont

17. What is PostgreSQL Low cost open source database Been around for about 15 years Runs on most Distributed OS; not on z/OS ACID Compliant (Atomicity, Consistency, Isolation, Durability) Why PostgreSQL Reduce software costs First choice for new apps No additional charge for using QMF DB2 Support Introduction to SQL Class Covers DB2, Oracle, PostgreSQL To work with PostgreSQL, V10 QMF requires Fix Pack 10 PostgreSQL

18. Download PostgreSQL driver Download… PostgreSQL…cont

19. Bounce started task after copying driver. PostgreSQL…cont

20. Only one driver needed for everyone…

21. Auditing: First, add jar file for Oracle: odbc6.jar Oracle

23. And how did we copy data from one RDBMS to another, such as PostgreSQL? …and select the data source Copying Data

24. Several other options, such as a QMF Proc: Copying Data…cont CONNECT TO DB2 DISPLAY TABLE Q.STAFF CONNECT TO “solidDB esb0stl2_dtl1(CPUE)” SAVE DATA AS ownerid.STAFF -- If run as a Scheduled Job it will take less local memory and CPU time -- than if done manually: returned results from DISPLAY are not rendered -- as a grid. Proc can also be parameterized: CONNECT TO &DatabaseSource DISPLAY TABLE &Table CONNECT TO &DatabaseTarget SAVE DATA AS &Table -- This would result in the new table having the same name as the old. -- Of course it could be like this SAVE DATA AS &OWNER.&TABLE as -- shown

25. Not possible with DB2 on z/OS: -512; STATEMENT REFERENCE TO REMOTE OBJECT IS INVALID To correct: - DB2 LUW - Federated Database - Nicknames - Or - QMF FOR WORKSTATION / WEB Client Joining Tables from Different RDBMSs

26. Create virtual data source (example using with Web Client) Joining Tables from Different RDBMSs…cont

27. Copy solidDB and PostgreSQL tables into virtual data source Joining Tables from Different RDBMSs…cont solidDB staff already added… Let’s add Org from PostgreSQL

28. Enter SQL and run as normal…. Joining Tables from Different RDBMSs…cont

29. A few thoughts about joining data: Subject to SQL supported by the database Data stored on physical disk Understand the amount of data being returned QMF will execute “SELECT *” from each table to join. Add predicate to minimize the amount of data being joined Joining Tables from Different RDBMSs…cont

30. Analytical Query: Stores and joins data in memory, not disk like Virtual Data Source For large tables, add a predicate “WHERE” Joining Tables from Different RDBMSs…cont

31. Analytical Query: No SQL is written, QMF handles this for you Joining Tables from Different RDBMSs…cont

32. Monthly CPU Stats via Job Scheduler…3 steps. 1.QMF Proc to run queries -Output will be Excel spreadsheet -Output will be mailed -Subject heading on email note Job Scheduler on QMF Web Client xlsx available in QMF 11

33. Monthly CPU Stats via Job Scheduler 2. Define new job schedule & calendar: Job Scheduler on Web Client…cont Monthly CPU Stats via Job Scheduler 2. Define new job schedule & calendar:

34. Monthly CPU Stats via Job Scheduler 3. Output is sent to user and QMF tracks job completion: Note: The QMF Workstation client uses job scheduler on your PC; the QMF Web client uses built-in scheduler Excel Spreadsheet Job Scheduler on Web Client…cont

35. IBM Stored Procedure DSNWZP & QMF Dashboard (Let the fun begin) DSNWZP: Used to retrieve zParms; ADDRESS DSNREXX "EXECSQL CALL DSNWZP USING DESCRIPTOR :INSQLDA " IF SQLCODE ¬= 0 THEN Do … Command ===>, 000245,SQLSTMT1 = "INSERT INTO D2INS01.ZPARM_TABLE", 000246, "VALUES ('"|| report_date ||"'", 000247, ||","||"'"||getssid ||"'", 000248, ||","||"'"||ssid ||"'", 000249, ||","||"'"||abexp ||"'", 000250, ||","||"'"||abind ||"'", 000251, ||","||"'"||accel ||"'",. QMF Dashboard & DSNWZP

36. Stored Procedure DSNWZP & QMF Dashboard, Cont. In the beginning…build a scene QMF Dashboard & DSNWZP…cont

37. Stored Procedure DSNWZP & QMF Dashboard, Cont. In the beginning…add buttons, tables, etc QMF Dashboard & DSNWZP…cont

38. Stored Procedure DSNWZP & QMF Dashboard, Cont. QMF Dashboard & DSNWZP…cont

39. Direct quote from one of our applications: “ Just the shared workspaces makes QMF for Workstation a valuable query tool for teams.” QMF Workspaces - Security Shared Workspaces: In Eclipse, workspaces are used to organize a collection of projects. In QMF for Workstation, users can create a workspace to house projects that hold the artifacts (data sources and repository objects) needed to build interactive visual tools, such as visual dashboards and reports. As the administrator you can choose to have your QMF for Workstation users work with repository objects from a workspace, or directly from the repository. “Admin View”

40. Automatically created by first access Can be created by Administrator Only viewable by those who have the authority Right click on objects within QMF to select “INCLUDE IN HOME WORKSPACE” QMF Workspaces – Security...cont “User View”

41. QMF Workspaces – Security…cont

42. QMF and the IBM DB2 Analytics Accelerator (Stored Procedures)

43. Double “click” for Lob Indicator QMF and the IBM DB2 Analytics Accelerator (Stored Procedures)…cont

44. By default QMF for Workstation and Web Client inherit the acceleration from for Query Acceleration DB2 Analytics Accelerator based on the DB2 for z/OS setting CURRENT QUERY ACCELERATION To modify IDAA Current Register: QMF and the IBM DB2 Analytics Accelerator (Stored Procedures)

45. QMF; enhanced to work with IDAA: Since the introduction of the Accelerator, SAVE DATA has failed. Due to the way SAVE DATA works, queries that use that QMF feature are not eligible to run in the Accelerator. IDAA - IDT….In-Database Transformation (IDT) Used by QMF to build tables stored only in IDAA. This now allows SAVE DATA to run in Accelerator. PI30376: NEW FUNCTION ON DB2 10 FOR Z/OSPI30376: NEW FUNCTION ON DB2 10 FOR Z/OS is the APAR that introduced accelerator-only tables along with in-database transformation (IDT) and temporal query offload…Save Data QMF and the IBM DB2 Analytics Accelerator (Stored Procedures)

46. Hidden Gems

47. Font size Hidden Gems

48. F11 on QMF for Web Client Hidden Gems…cont

49. Entity Relationship Diagram; many features in QMF, like Prompted Query, but ER Diagrams visually show it. Hidden Gems…cont

50. Password Propagation Hidden Gems…cont

51. What SSID am I working on? Hidden Gems…cont

52. Content Assist: Content Assist helps to complete SQL statements by providing lists of options while the statement is being typed. Ctrl + Space Bar The contents of the drop- down list depend on the location of the cursor in the SQL statement and the contents of the table being called. Hidden Gems…cont

53. Content Assist… Hidden Gems…cont Ctrl + Space Bar

54. Content Assist… Ctrl + Space Bar Hidden Gems…cont

55. Format SQL Text: (Not available on Web) Hidden Gems…cont

56. Color Text: The SQL Query editor provides coloring support for your SQL statements. (not available with Chrome) Hidden Gems…cont

57. Multiple Windows Hidden Gems…cont

58. Running multiple queries Hidden Gems…cont

59. No need to click the green arrow… (Ctrl + R) or F2 to execute Hidden Gems…cont

60. Reset Perspective; Eclipse can be confusing, get your view back Hidden Gems…cont

61. Save DB2 CPU….Cache Prepare: Alt+Q+P just does a PREPARE and validates the query without the overhead of executing it and returning the result. Note: If a user runs the same query more than once during the same caching interval (2 hours here) then QMF leaves the database alone and uses the locally cached data.

62. Hidden Gems…cont Charts….you don’t need Excel

63. Troubleshooting: Viewing Errors Hidden Gems…cont

64. Troubleshooting: Collecting data for IBM Hidden Gems…cont

65. youtube.com SG24-8012-00; Complete Analytics with QMF Getting Started with QMF for Workstation and Web Client developerWorks.com (search for QMF) Places to visit:

66. “A special thank you to Shawn Sullivan of Rocket Software for his assistance over the many years.” Thank You

67. © 2015 IBM Corporation Thank You