1 2002: The Year of Archiving Robert Parkinson PCS Limited “Creating Solutions Through Knowledge”

2 What We’ll Cover Archiving benefits Definitions of terms Archiving features Archiving steps Storage and management Archiving by module What next?

3 What is Archiving? Archiving Program Archive Files R/3 Database Move data no longer required in database to archive files Archiving Session

4 Why Archive? Volume of data in the database increases Disk space Performance Size of index trees, full table scans, cache hit ratios,... Effort and resources required for database administration Backup/restore, reorganization, create index, statistics,... Business and legal requirements Auditing Record-keeping requirements Potential re-useability of data

5 Data Continues to Grow Yearly data growth 400 GB, retention period 6 months Data not needed online after 5 years: 1,800 GB Volume without data archiving in 5 years: 1,000,000 GB Volume with data archiving in 5 years: 225,000 GB Backup, restore and database administration time grows proportional to database size without data archiving

6 Archiving Requirements Independence of database version of version and customizing of application software of hardware (codepage / numeric format) Auditability Revenue authorities – IRS, auditors Data security officials Integration in applications Direct access Reporting functions Performance and robustness

7 What We’ll Cover Archiving benefits Definition of terms Archiving features Archiving steps Storage and management Archiving by module What next?

8 Optical Archiving l ArchiveLink can be used to archive documents l Original documents or print lists can be stored in optical archive systems and displayed, but cannot be reloaded or analyzed l Archive files which were created by R/3 data archiving can be stored, read, and reloaded using ArchiveLink

9 Database Reorganization l Should be performed after archive l Compelling reasons: Tables keep free space allocated after archive Gives free space back to the database Improves performance Reduces database maintenance times

10 Retention vs. Residence Period l The retention period is the entire time that data spends in the database before its archived l The residence period is the minimum length of time that data must spend in the database before its eligible to be archived

11 Backup & Restore l Contains a copy of the database contents for security l Backups are usually made at regular intervals, according to a standard procedure l Reloading the backup into the database is referred to as restore l In earlier SAP releases, the archiving procedure required a database backup. SAP states this is no longer necessary in Release 3.X and 4.X

12 What We’ll Cover Archiving benefits Definition of terms Archiving features Archiving steps Storage and management Archiving by module What next?

13 Archiving Features High data security during archiving Optical archive connection via ArchiveLink Data compression Online data archiving Archiving without database backup Random and sequential access to archived data objects, reload to the database possible Automatic conversion of old archive files The ADK (Archive Developer’s Toolkit)

14 Security Provided in Archiving l Archiving is a two step process: l Create archive files l Read data from file and execute delete program l Storage of the archive files on other media l Provides data security in case of problems during the archiving process

15 Step 1 Create the Archive File R/3Data-base Archiving program Archivefile

16 Step 2 Execute Delete Program R/3Data-base Archiving program Archivefile Delete program Archivefile R/3Data-base Archivefile Delete program Archivefile Delete program

17 Step 3 Store the Archive Files R/3Data-base Archivefile Archivefile Delete program ArchiveLink HSM-Systems

18 Optical Archive Link ArchiveLink The archive files can be stored on tertiary storage media Files can be stored manually or automatically Hierarchical storage management systems (HSM systems) Archive files can also be passed to an HSM system HSM systems can use optical and optomagnetic storage media The HSM system manages the archive files automatically ArchiveLink R/3 System Archive system

19 Compression Data is automatically compressed ( Up to a factor of 5 or greater) Online data archiving lData archiving may run in parallel to dialog and batch processing Data archiving without database backup lData archiving is fully recoverable

20 Access to Archived Data l There are two ways to access archived data: Sequential access to a dataset (reporting) Random access to individual data objects l Archive files can be Read Analyzed (for reporting purposes) Reloaded (Not Recommended – only right after archiving!) l Actions depend on archiving object l Access for reporting and reloading requires that the file is in the file system l Access to individual objects depends on an index

21 Conversion of Old Archive Files Necessary conversions are performed automatically Temporary conversion during data access In rare cases, conversion programs are provided

22 The ADK Databas e R/3 System LDBLDB ABAP/4 Program Archiving Class Codepage, Number Format, Structure Conversion Direct Access, Compression, Batch-Handling,.... ADK: Archive File manually HSM Archive System with Tertiary Storage Media ArchiveLink

23 What is an Archiving Object? R/3 System Database Archive Key to archiving concept Description of database objects Collection of programs

24 Customization During Data Archiving Creating an archiving object Define archive object using transaction AOBJ Create programs for archiving Archive program Delete program Reload program Index program Create programs for accessing archived data Reports Direct access via online transaction

25 What We’ll Cover Archiving benefits Definition of terms Archiving features Archiving steps Storage and management Archiving by module What next?

26 Archiving Personnel l Archiving can be performed by the Basis Administrator by the application department l Motivation (and point of entry) for both are different l Basis Administrator The Basis Administrator checks the database/table size Archiving object relationship has to be determined l Application Dept. Application Dept. performs archiving for various reasons (periodically – month end, quarter end, year end) Application Dept. uses direct point of entry

27 Archiving Flowchart

28 Identifying Critical Tables Database Performance Screen (DB02) Overall database growth analysis Determine size of tablespaces and tables Free storage space – allocated vs used space Table Call Statistics Screen (ST10) Locate the most active tables Which tables are growing Which tables have slow response

29 Identifying Critical Archiving Objects Use archive administration (SARA) Browse through the archiving objects Determine which tables are in the archiving objects Schedule all required archive programs Use CCMS database administration (DB15) Determine which archiving objects use a table Check online space information for all tables of an archiving object

30 Business Requirements for Archiving Objects (SD_VBAK) Document has status "Complete" All documents in document flow have status "Complete" Financial document is balanced Additional customized checks possible

31 What We’ll Cover Archiving benefits Definition of terms Archiving features Archiving steps Storage and management Archiving by module What next?

32 Storage & Management Automated storage Low management effort Manual storage Data medium maintenance Data medium management Archive production

33 Storage of Archive Files Choice of medium is critical for: Utilization of the functionality offered by SAP Effort required for maintenance Effort required to access data Durability of archive  Protection of investment by means of combined methodology Data archive files Workflow for incoming and outgoing records DMS (Document Mgmt. System) technical documents Printed reports Automation of database backup and restore

34 Archive Files on Hard Disk Advantages Single document access < 5 seconds Quick access for reporting Disadvantages File system limitations Considerable manual management effort required Replicate the archive files Storage of archive files Data security

35 Archive Files on HSM System Advantages Single document access < 30 seconds (with block access) Quick access for reporting (with block access) Scalable file system Long data medium retention period Data medium management support Disadvantages Can be a costly solution

36 HSM System Requirements "Write through" should be possible Increased data security, as the archive files migrate before the delete program runs Data retained in "magnetic cache" as copy "Random access" to files must be possible "Hardware sharing" is useful Use of a jukebox via both HSM and ArchiveLink "Media management" should be supported Replicate archive files Store archive files

37 Archive Files on Jukebox via ArchiveLink Advantages Scalable system Long data medium retention period Data medium management support Jukebox can also be used for other R/3 tasks Disadvantages Can be a costly solution Single document access only feasible for small archive files Access time heavily depends on archive file size

38 ArchiveLink Capabilities ArchiveLink R/3 System Archive system With ArchiveLink you can: Archive incoming R/3 documents Archive information created in R/3 Directly access the optically archived documents from the R/3 applications Integrate an optical archive system with R/3 Replace your existing paper archive with electronic archive systems

39 Some Considerations for Media Jukebox via ArchiveLink Archived data seldom accessed individually Access time is not critical, random access slow in Rel. < 3.1G HSM system with block access Archived data are often accessed individually Access time can be on the order of 30 seconds Hard disk Archived data are regularly accessed individually Access time must be on the order of 5 seconds Magnetic tape Archived data are only accessed in exceptional cases

40 Sample Access Times for Archived Data Sec.

41 What We’ll Cover Archiving benefits Definition of terms Archiving features Archiving steps Storage and management Archiving by module What next?

42 FI Documents 600,000 FI documents per month Retention period up to 18 months 120 GB used space in DB for FI data 100,000 documents archived in 3 hours, 80 MB archive file  Reduce retention period to 3 months  Data archiving by IT organization

43 MM Documents 450,000 MM documents and 2,500,000 line items per month Line-item table bigger than 10 GB 100,000 line items being archived in 3 hours Direct access implemented by customer  Reduce retention period  Random access and reporting on data archive files shows good performance

44 Access Times … Example Optimal situation, single job, elapsed time

45 What We’ll Cover Archiving benefits Definition of terms Archiving features Archiving steps Storage and management Archiving by module What Next?

46 What next? Increase in third party solutions Better reporting for archiving objects More robust document imaging solutions Enhanced performance of ADK Further integration of archiving and database administration

47 Sources of Information Check OSS for your platform SAPNET good source of information SAPLabs, SAP Search, mySAP.com SAP’s Early Watch for data archiving

48 Your Turn! (to launch Q&A) PCS Limited, A Professional IT Consulting Services Company “Creating Solutions Through Knowledge” “Creating Solutions Through Knowledge” PCS Limited 5425 Parkford Circle Granite Bay, CA USA Granite Bay, CA USA Tel Fax