1. Express Reorg: How to save your weekend Laura Rochon Ajilon Consulting

2. Laura Rochon Started working with CA-IDMS in 1983 Joined Cullinet in 1985 Left CA in 1992, has been consulting ever since Joined Ajilon Consulting in January 2005 IUA Board member since 1998 CQUI President

3. Agenda Overview of Unload/Reload Overview of Express Reorg Express Reorg Execution Considerations Test results Conclusion

4. Overview of Unload/Reload UnloadReload DB

5. Overview of Unload/Reload UnloadSORT1DBL2 SORT2DBLX SORT3DBL3SORT4DBL4 ReloadUnload DB

6. Overview of Express Reorg DB UnloadReload DB Unload Reload Control file

7. Overview of Express Reorg New functionality: – Add or remove fields from index key – Change order of set

8. Overview of Express Reorg 4 major phases: – SETUP Allocates control file Specifies what is to be done – UNLOAD Unloads data – RELOAD Reloads data – CLEANUP Delete work files (but not control file) Unlock areas

9. Overview of Express Reorg Unload: – Area sweep for most records, except: Record stored VIA system-owned index VIA record CALC records with duplicate option

10. Overview of Express Reorg Unload (cont’d) – Records assigned target page CALC records VIA records Direct records

11. Overview of Express Reorg REORG slicing: – Based on DIVIDE PROCESSING n WAYS – Generally, each area is divided symmetrically – If total size of all areas is less than 200 pages, 1 slice – Areas and subareas that contain only system indexes not divided and assigned to a slice

12. Overview of Express Reorg REORG slicing (cont’d) – User-owned indexes are processed with the slice of the owner record – System-owned indexes assigned to index group and processed indepently of slices

13. Overview of Express Reorg Reorg processing is divided into tasks and grouped into phases RELOAD phase is divided into – Reload phases 1 thru 6 Each reload task in a given task must successfully complete before processing can move to next Reload phase – Rebuild phases 1 thru 4

14. Overview of Express Reorg Unload Rebuild 1 REORG: Reload 1 Reload 2 Reload 3 Reload 4 Reload 5 Reload 6 Rebuild 2 Rebuild 3 Rebuild 4 Cleanup Reload Unload Cleanup DB

15. Express Reorg Execution Parameters JCL Work Files Output Processing Job submission

16. Express Reorg Parameters SEGMENT source-segment USING source-ssc AREA area RELOAD INTO target-db USING target-ssc DMCL dmcl-name DIVIDE PROCESSING n WAYS REUSE WORKFILES AS SORTEXIT SHARE NOTIFY notify-interval REORG setup options :

17. Express Reorg Parameters REORG execution options: CLEANUP STOP AFTER SETUP UNLOAD RELOAD NO CREATE UNLOAD WORKFILES RELOAD ALL SUBMIT

18. Express Reorg Parameters REORG other parameters : CLEANUP STATUS REPORT ONLY

19. Express Reorg JCL //BCF EXEC PGM=IDMSBCF,REGION=0M //STEPLIB DD DSN=idms.loadlib,DISP=SHR //SYSIDMS DD * Sysidms parameters //SYSLST DD SYSOUT=* //SYSPRINT DD SYSOUT=* //SORTMSG DD SYSOUT=* //SORTWRKnn DD UNIT=unit,SPACE=(CYL,(mmm,nnn)) //SYSIPT DD * Reorg parameters //RORGCTL DD DSN=rorgctl-file,DISP=SHR //RORGJCL DD DSN=jcl-file,DISP=SHR

20. Express Reorg JCL RORGCTL file: – Describes the specifics of the REORG operation – Contains status information – Fixed-length, LRECL=4096 – Initialized during setup – Should not have DISP=OLD or DISP=NEW

21. Express Reorg JCL RORGJCL file: – Contains JCL for parallel jobs – Job name should be short enough to append 1+ characters – 1 step: IDMSBCF with following syntax: REORG; – Should have DB files and RORGCTL file

22. Express Reorg Work Files The number of work files proportional to number of jobs submitted Work files can be manually allocated Work files allocated automatically by using new command CREATE DSMODEL Can specify several CREATE DSMODEL

23. Express Reorg Work Files REORG uses following DDNAME prefixes: – WU -- unload/reload work files – WI -- Index Rebuild work files – WS-- Sort output work files – WD-- DBKEYS work files

24. Express Reorg Work Files CREATE DSMODEL creates a temporary model of data set attributes Must include information : – Where to create work file – Space – Blocksize

25. CREATE DSMODEL

26. Express Reorg Work Files Sizing work files: – Very hard to estimate – Run REORG STOP AFTER UNLOAD with dummied work files.

27. Express Reorg Work Files Number of files grows exponentially: example of WU* files: Divide n waysNo reuseReuse workfiles 1119 23224 36345 410472 5155105

28. Output

29. Output

30. Output

31. Output

32. Output

33. Output

34. Output

35. Output

36. Output

37. Express Reorg Processing If Target files same as Source files – REORG STOP AFTER UNLOAD – If not SHARE, UNLOCK AREAS – FORMAT files – REORG with STOP AFTER CLEANUP & SUBMIT – Backup – IDMSDBAN (optionally) – PRINT SPACE (optionally)

38. Express Reorg Processing If Target files with different DDnames – Format new files – REORG with STOP AFTER CLEANUP – Backup – PRINT SPACE – IDMSDBAN (optionally)

39. Express Reorg Processing If Target files different but same DDnames – Format new files – REORG with STOP AFTER UNLOAD (REORG jcl and jcl in RORGJCL have old DDnames) – REORG with STOP AFTER CLEANUP and SUBMIT (REORG jcl and jcl in RORGJCL have new files)

40. Job Submission Automatic job submission Manual job submission Restarting REORG

41. Automatic Job Submission Jobs automatically submitted – at end of SETUP phase – If SUBMIT parameter is specified REORG will submit – 1 job for each slice – 1 job for each index group Maximum jobs submitted = 2x no slices

42. Manual Job Submission Can manually submit REORG job(s) Job will examine control file and determine if there is work to be done If no work, waits until some other job completes a task If work is needed, it performs that work

43. Restarting REORG If a REORG job fails, other jobs will try to restart the task that failed If problem is not temporary (work file too small), problem needs to be corrected, then REORG can be submitted with SUBMIT. In some severe cases, you might need to restart REORG from the beginning

44. Considerations Relative REORG performance Efficiency of the reorganized database Work files

45. Relative REORG performance Usually runs in less time than normal UNLOAD/RELOAD, however….. Best to try it out first

46. DB Efficiency DB might not be as efficient as reorganized by UNLOAD/RELOAD REORG reloads in forwards fashion whereas UNLOAD/RELOAD reloads in backwards fashion REORG can result in more CALC & VIA overflow especially if long sets

47. Work Files Work files HAVE to be on disk Hard to estimate their size

48. Tests Results 67.71 33.54 28.16 24.47 24.04 Unload/reload Reorg 2 waysReorg 3 waysReorg 4 waysReorg 5 ways 50.5% 58.4% 63.8% 64.5% TSTDML (46,116 pages)

49. Tests Results 40.9 29.78 22.89 Unload/reload Reorg 2 waysReorg 3 ways 27.2% 44.0% 1 Area unloaded (120,000 pages going to 147,000 pages) 1 Area affected (45,000 pages)

50. Tests Results 1 Area unloaded (427,720 pages @5064 going to 428,400 pages @ 7548) 3 Areas affected (39,000 pages 160,000 pages 294,000 pages) 168.27 89.31 80.23 46.9% 51.1% Unload/reload Reorg 3 ways Reorg 5 ways

51. Conclusion Can reduce the time to reorganize a database up to 50-60% Easy to use Needs lots of disk space