1. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 1 Patterns for Effective Management of Resources in Software Systems Presented by Douglas C. Schmidt Based on the POSA3 book by Michael Kircher Prashant Jain Reproduced with permission from Pattern-Oriented Software Architecture, Volume 3: Patterns for Resource Management, Michael Kircher & Prashant Jain. © 2004 John Wiley & Sons Ltd.

2. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 2 Agenda Introduction Resource Acquisition Resource Lifecycle Resource Release Case Studies Summary

3. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 3 What is Resource Management? A resource is an entity that is available in limited supply e.g., memory, CPU, network bandwidth, etc. A resource user is an entity that acquires, accesses, or releases resources e.g., an application or system program A resource provider is an entity that provides the resources on request e.g., a virtual memory manager, thread scheduler, network interface, etc. Resource management in software systems is the process of controlling the availability of resources to resource users. Resource management ensures that resources are available when needed, that their lifecycle is deterministic, & that they are released in a timely manner to ensure the liveliness of the systems that use them.

4. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 4 Resource Categorization By resource lifecycle Reusable: Resources once released can be acquired & used again oExamples: Memory, threads, file handles Non-reusable: Resources are consumed, & therefore once acquired are either not released, or their release is implicit oExample: Processing time in a computing grid By resource usage Exclusive: Resources that can only be used by a single user at a time Example: Processing time of a service Concurrent: Resources that can be concurrently accessed by multiple users Examples: Queues & databases ReusableNon-reusable ExclusiveMemoryProcessing-time ConcurrentRead-only object

5. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 5 Forces behind Effective Resource Management (1 of 2) Performance High-performance systems require low latency & high throughput. Therefore: Unnecessary resource acquisitions, releases, & accesses that incur processing overhead & delay must be avoided. Scalability Large & complex systems may require support for more users or higher transfer volumes. Therefore: The way resources are managed & the way their lifecycle is managed must be carefully controlled. Predictability In systems with real-time requirements, the maximum response time of individual operations must be predictable. Therefore: Acquisition & release times must not vary.

6. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 6 Forces behind Effective Resource Management (2 of 2) Flexibility Ease of configuration & customization is a common requirement among systems. Therefore: The mechanics of acquisition, release & lifecycle control of resources need to be flexible, while still preserving performance, reliability, & scalability requirements. Stability An important requirement of software systems is that frequent resource acquisition & release should not make the system unstable. Therefore: Resource allocation & management must be done in a manner that leads to efficient use of resources & avoids scenarios of resource starvation that can lead to system instability. Consistency All resource acquisition, access, & release must leave a software system in a consistent state. Therefore: Any inter-dependencies among resources must be well-managed to ensure system consistency.

7. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 7 What are Design Patterns? Patterns facilitate reuse of successful software architectures & designs Patterns identify & specify abstractions that are above the level of single classes & instances Patterns capture the static & dynamic structure & collaboration among key participants in software design Patterns provide a common vocabulary & understanding for design principles A design pattern describes a solution for a recurring problem that arises in a particular context The Proxy Pattern 11 Proxy service Service service AbstractService service Client

8. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 8 The Three Main Ingredients Context: a situation giving rise to a problem Describes when it makes sense to apply the pattern Problem: the recurring problem arising in the context Identifies the forces that must be resolved & identifies any constraints that must be considered Solution: a proven resolution of the problem Provides a means to balance most (if not all) of the forces Describes the key steps required to address the problem

9. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 9 The Three Main Ingredients Applied to Proxy The Proxy Pattern 11 Proxy service Service service AbstractService service Client Context: when developing a distributed application Problem: need to alleviate tedious & error-prone differences in byte-orders & alignment constraints across heterogeneous platforms Solution: define a proxy that performs (de)marshaling & shields applications from network programming details

10. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 10 Pattern-oriented Effective Resource Management Resource Lifecycle Manager Caching Pooling Leasing Evictor Partial Acquisition Lazy Acquisition Coordinator Lookup Eager Acquisition Performance Scalability Consistency Stability Flexibility Predictability

11. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 11 Patterns for Resource Acquisition “I find that a great part of the information I have was acquired by looking up something and finding something else on the way.” Franklin P. Adams

12. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 12 Lookup – Example A system consisting of multiple distributed services To access a service, client needs a reference to the object that provides that service For example, a client would need a reference of the Transaction Manager to participate in distributed transactions How can a client obtain the initial reference to an object that the client wants to access? Network Client Transaction Manager get object reference publish object reference

13. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 13 Lookup Context Systems where resource users need to find & access local & distributed resources Problem How can a resource user find out on demand what resources are available in its environment? How can a resource user obtain an initial reference to a resource provider that offers the resource? Solution Provide a lookup service that allows resource providers to make resources available to resource users The resource provider advertises resources via the lookup service along with properties that describe the resources that the resource providers provide Allow resource users to first find the advertised resources using the properties, then retrieve the resources, & finally use the resources Lookup Service Resource Resource User Resource Provider

14. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 14 Lookup – Example Resolved The server that creates the Transaction Manager should register the reference of the created Transaction Manager with the Lookup Service. The server is therefore the resource provider. In CORBA, the Lookup Service is implemented by the Naming Service. A client can obtain the object reference of the Transaction Manager from the Lookup Service.

15. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 15 Known Uses: Lookup CORBA The Common Object Services Naming Service & Trading Service implement lookup services. Jini Jini services are registered with Jini’s lookup service, & these services are accessed by users using Jini’s discovery protocol. COM+ The Windows Registry is a lookup service that allows resource users to retrieve registered components based on keys. Universal Description, Discovery, & Integration protocol (UDDI) UDDI allows publishers of Web Services to advertise their service & clients to search for a matching Web Service. Grid Computing Grid computing uses the Lookup pattern to find distributed resources. Eclipse plug-in registry The plug-in registry holds a list of all discovered plug-ins, extension points, & extensions & allows clients to locate these by their identity.

16. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 16 Lazy Acquisition – Example How can the PACS system be designed so that retrieval of patient information is quick regardless of the number of images in the patient’s record? Picture Archiving & Communication System (PACS) that provides storage of patient data Data includes patient details, such as address information, as well as digitized images & is represented by business objects For a patient with a long medical history & several digitized images, fetching all the data & creating the corresponding business object(s) can take a lot of time

17. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 17 Lazy Acquisition Context A system with restricted resources that must satisfy high demands, such as throughput & availability Problem Reduction of the initial cost of acquiring the resources If systems were to acquire all resources up front, a lot of overhead would be incurred & a lot of resources would be consumed unnecessarily Solution Acquire resources at the latest possible time The resource is not acquired until it becomes unavoidable to do so Introduce a resource proxy that hides the lazy acquisition from the resource user Resource User Resource Resource Proxy Resource Provider

18. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 18 Lazy Acquisition – Example Resolved Do not fetch any image data, when fetching information for a particular patient Create an image proxy for each image in the patient record returned When an image is actually viewed, it is fetched (lazily) from the image store Specializations: Lazy Instantiation Lazy Loading Lazy State

19. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 19 Lazy Acquisition – Known Uses Singletons Objects that exist uniquely in a system, are usually instantiated using Lazy Instantiation J2EE EJB containers use Lazy Loading & Lazy Instantiation to ensure that only components that are actually used by clients are active Java Server Pages (JSPs) are typically compiled into servlets only when they are actually accessed, rather than when they are deployed.NET Remoting Singleton remote objects are instantiated only on first access, even though clients might obtain references to them before the first access JIT compilation The compilation of the regular Java byte code into fast machine-specific assembler code is done just-in-time Eclipse plug-in Actual logic of a plug-in, which is contained in Java archives (JAR), is loaded lazily on first use of any of the plug-in’s functionality

20. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 20 Eager Acquisition – Example Embedded telecommunication application with soft real-time constraints, such as predictability & low latency in execution of operations In most operating systems, operations such as dynamic memory allocation can be very expensive Memory allocations are protected by synchronization primitives Memory management, such as compaction of smaller memory segments to larger ones, consumes time How can you achieve predictable & fast memory allocations via operations such as new() or malloc()?

21. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 21 Eager Acquisition Context A system that must satisfy high predictability & performance in resource acquisition time Problem Expensive resource acquisitions, such as dynamic acquisition of threads & memory, can result in unpredictable time overheads How can resources be acquired in systems with soft real-time constraints while still fulfilling the constraints? Solution Eagerly acquire a number of resources before their actual use At a time before resource use, optimally at start-up, the resources are acquired The resources are then kept in an efficient container Resource User Resource Provider Proxy Resource Provider

22. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 22 Eager Acquisition – Example Resolved Implement a simple memory pool for fixed-size blocks of memory without synchronization to be used by a single thread Eagerly acquire the memory for the pool Provide separate memory pools for different ranges of memory size Solution allows for a ’natural’ & flexible control flow, while still ensuring predictability class Fixed_Size_Memory_Pool { public: Fixed_Size_Memory_Pool (size_t max_blocks) { //... Eagerly allocate free list blocks... } void *acquire (size_t size) { //... void *acquired_block = free_list_.front (); free_list_.pop_front (); return acquired_block; } void release (void *block) { free_list_.push_back (block); } };

23. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 23 Eager Acquisition – Known Uses Ahead-of-time compilation Ahead-of-time compilation is commonly used by Java virtual machines to avoid compilation overheads during execution Pooling Connection or thread pools, typically pre-acquire a number of resources, such as network connections, or threads, to serve initial requests quickly Application servers An element called can be specified for a servlet in the deployment descriptor, causing the container to load that servlet at start-up. Eclipse plug-in Plug-in declarations that determine the visualization are loaded eagerly The actual logic, which is contained in Java archives (JAR), is loaded lazily on first use of any of the plug-in’s functionality Hamster A hamster acquires as many fruits as possible before eating them in its burrow

24. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 24 Partial Acquisition - Example Network Management System (NMS) that manages several network elements (NEs) NMS allows user to get details of one or more NEs Details of an NE may correspond to a large amount of data The details of all the NEs can be fetched at system start-up or recovery Since obtaining all the details of the network elements can have a big impact on the time it takes for the application to start up or recover, how do we make the application start-up or recovery time more efficient & predictable? GUI NE Shelf Card Network Management System

25. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 25 Partial Acquisition Context Systems that need to acquire resources efficiently. The resources are characterized by either large or unknown size Problem In highly robust & scalable systems, how can it be ensured that resources are acquired efficiently? How can acquisition of resources be influenced by parameters such as available system memory, CPU load, & availability of other resources? Solution Split the acquisition of a resource into two or more stages. In each stage, acquire part of the resource The amount of resources to acquire at each stage should be configured using one or more strategies Eager Acquisition & Lazy Acquisition can be used to determine when to execute one or more stages to partially acquire a resource Resource Provider Resource Resource User

26. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 26 Partial Acquisition – Example Resolved Acquisition of the details of the NEs, along with their components, can be split into multiple stages In the initial stage only the details of the NEs are fetched In subsequent stages, the details of the components of the NE are fetched The details can be fetched in the ‘background’ using Active Object pattern Partial acquisition of the details of an NE can also trigger the retrieval of the neighboring NEs

27. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 27 Partial Acquisition – Known Uses Incremental image loading in web browsers Browsers first download the text content of a Web page The images are downloaded incrementally, during which time the user can read the text content of the page Socket input Multiple read operations are performed at a socket, where each read operation partially acquires data from the socket & stores it in a buffer Audio/Video streaming When decoding audio & video streams the streams are acquired in parts Data-driven protocol-compliant applications An application handling CORBA IIOP requests typically reads the IIOP header in the first step to determine the size of the request body It then reads the contents of the body in one or more steps

28. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 28 Patterns for Resource Lifecycle “Seek not, my soul, the life of the immortals; but enjoy to the full the resources that are within thy reach.” Pindar

29. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 29 Caching – Example Network management system that needs to monitor the state of many network elements Establishing a new network connection for every network element selected by the user & destroying it after use incurs overhead in the form of CPU cycles The average time to access a network element can be too large How can the average time to access a network element be optimized, while also reducing the number of network connections that are created & destroyed?

30. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 30 Caching Context Systems that repeatedly access the same set of resources & need to optimize for performance Problem Repetitious acquisition, initialization, & release of the same resource causes overhead How can you reduce the acquisition, access, & release cost of frequently used resources to improve performance? Solution Temporarily store the resource in a fast-access buffer called a cache Use the cache to fetch & return the resource instead of re-acquiring it The cache identifies resources by their identity, such as pointer, reference, or primary key Resource User Resource Resource Cache Resource Provider

31. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 31 Caching – Example Resolved Implement a cache of connections of network elements On user access of a specific network element, the connection to the network element is acquired When the connection is no longer needed, it is not destroyed but is added to the cache Later, when a new request for the connection arrives, the connection is acquired from the cache, thus avoiding high acquisition cost Variants Transparent Cache Read-Ahead Cache Cached Pool

32. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 32 Caching – Known Uses Data transfer object An object that is transferred by value in middleware technologies such as CORBA & Java RMI is held locally in a cache & represents the actual remote object Web proxy A Web proxy is a proxy server located between a Web browser & Web servers The Web proxy caches a Web page that is fetched by a browser from a Web server, & returns it when a subsequent request is sent for the same page.NET Data Sets.NET data sets are instantiated locally & filled by the results of an SQL query on the database with one or more tables, using an SqlDataAdapter The data sets therefore provide caching of data, thus saving expensive database queries Enterprise JavaBeans (EJB) EJB entity beans represent database information in the middle tier The entity beans therefore help avoid expensive data retrieval (resource acquisition) from the database

33. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 33 Pooling – Example Simple Web-based on-line commerce application providing a catalog Contents of catalog dynamically generated from database Servlet connects to database, executes a query & uses results to generate HTML page Catalog can be accessed by hundreds of users simultaneously How do we ensure that a reasonable number of database connections are created & furthermore a database connection is available quickly to service a user request? Servlet A Database Servlet B Servlet Engine connections

34. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 34 Pooling Context Systems that continuously acquire & release resources of the same or similar type, & have to fulfill high scalability & efficiency demands Problem How can released resources be reused to avoid the overhead of re-acquisition? How can the acquisition time of a resource by a user be made predictable even though direct acquisition of the resource from the resource provider can be unpredictable? Solution Manage multiple instances of one type of resource in a pool. This pool of resources allows for reuse when resource users release resources they no longer need The released resources are then put back into the pool When a resource is released & put back into the pool, it should be made to lose its identity Resource User Resource Resource Pool Resource Provider

35. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 35 Pooling – Example Resolved Database connections are pooled in a connection pool On every request by the user, the servlet acquires a connection from the connection pool, which it uses to access the database After access, the connection is released to the connection pool

36. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 36 Pooling – Known Uses JDBC Connection Pooling Java Database Connectivity (JDBC) connections are managed using connection pooling COM+ The COM+ Services runtime, as part of the Windows operating system, supports object pooling for COM+ objects &.NET applications EJB Application Servers Component-based Application servers use component instance pooling to manage the number of component instances efficiently Web Servers Most Web servers use Thread Pooling to efficiently manage the threads Threads are reused after completing a request

37. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 37 Coordinator – Example Large-scale system distributed over multiple processing nodes Each node comprises of services that need to be frequently updated Successful update of some sub-systems but not all can leave the system in inconsistent state How do we ensure that either all the subsystems are successfully updated or none are? Sub-System-1 Sub-System-n Sub-System-2 Central Admin Component update

38. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 38 Coordinator Context Systems where a task needs to be coordinated among multiple participants Problem How do we ensure that in a task involving two or more participants, either the work done by all of the participants is completed or none is? Solution Introduce a Coordinator that is responsible for execution & completion of a task Split the work of each participant into two phases: prepare & commit In first phase, Coordinator asks each participant to prepare for the work to be done If prepare phase is successful, Coordinator initiates commit phase, in which the actual work is done by the participants If prepare phase fails, Coordinator aborts the task Coordinator Participant * 1 Coordination prepare() commit() abort() beginTask() register() commitTask()

39. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 39 Coordinator – Example Resolved Each sub-system acts as a participant & implements the Coordination interface The task to be completed is software update Coordinator executes the two-phase execution of the task

40. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 40 Coordinator – Known Uses Java Authentication & Authorization Service (JAAS) Login Context serves the role of Coordinator Login Modules are participants that follow two-phase process for authentication Java Transaction Services (JTS) JTS Transaction Manager serves the role of Coordinator Software Installation In prepare phase, checks are performed, such as making sure that there is enough disk space Databases Make use of two-phase commit to ensure data consistency Priest conducting a marriage Priest is the Coordinator, who follows a two-phase process Bride & groom are the participants

41. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 41 Resource Lifecycle Manager – Example Distributed system that needs to service thousands of clients Clients establish connections to communicate with remote services The complexity in managing the lifecycles of connections can affect the core functionality of the application Resource management can make its business logic hard to understand & maintain due to the entangled connection-management code How can the lifecycle of interdependent resources be managed while ensuring that the business logic of a system does not become too hard to understand & maintain?

42. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 42 Resource Lifecycle Manager Context Large & complex systems that require simplified management of lifecycle of their resources Problem Resources in systems can be of many different types, such as network connections, threads, synchronization primitives, servants & so on Resources of the same or different types might depend on each other How can the lifecycle of interdependent resources be managed such that it does not make the business logic of the system hard to understand & manage? Solution Separate resource usage from resource management Introduce a separate Resource Lifecycle Manager whose sole responsibility is to manage and maintain the resources of a resource user The Resource Lifecycle Manager maintains the dependencies between resources Resource User Resource Resource Lifecycle Manager Resource Provider

43. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 43 Resource Lifecycle Manager – Example Resolved Introduce a component that is responsible for the resource lifecycle management & hence free the application from this responsibility The clients then use the resource lifecycle manager to request new connections to the server Once these connections are given to the clients, their lifecycle is managed by the resource lifecycle manager public ResourceLifecycleManager { // resource group management public ResourceGroup createGroup (String groupID, Resource resources []) { //... } public void addResourceToGroup (Resource resource, ResourceGroup group) { //... } public boolean release (ResourceGroup group) {//... } //... }

44. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 44 Resource Lifecycle Manager – Known Uses Component container A component container manages the lifecycle of application components & provides application-independent services Further, it manages the lifecycle of resources used by the components Examples include: J2EE Enterprise JavaBeans (EJB), CORBA Component Model (CCM), & COM+ Remoting middleware Middleware technologies such as CORBA &.NET Remoting ensure the proper lifecycle management of resources such as connections, threads, synchronization primitives, & servants implementing the remote services Grid computing A resource lifecycle manager is typically used to manage the local resources of computers participating in a grid Resource lifecycle manager ensures that resources in a grid are available & fulfills local as well as distributed requests to those resources

45. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 45 Patterns for Resource Release “I saw the angel in the marble and carved until I set him free.” Michelangelo Buonarroti

46. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 46 Leasing – Example Quoter service, which provides stock quotes to clients Reference of Quoter service is made available to clients via Lookup service Suppose the Quoter service were to crash & not come up again Quoter service reference would become invalid but would still be available via the Lookup service How can the invalid Quoter service reference be automatically removed at the Lookup service? Network Client Quoter Service query object reference publish object reference Lookup Service

47. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 47 Leasing Context Systems where resource use needs to be controlled to allow timely release of unused resources Problem How can resources that are not used by a resource user, or no longer available be automatically freed as soon as possible to make them available to new resource users? How can the system load caused by unused resources be minimized? How can it be ensured that a resource user does not use an obsolete version of a resource when a new version becomes available? Solution Introduce a lease for every resource that is held by a resource user A lease is granted by a grantor & is obtained by a holder A lease specifies a time duration for which the resource user can hold the resource Once the time duration elapses, the lease is said to have expired & the corresponding resource is released LeaseGrantorResourceHolder

48. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 48 Leasing – Example Resolved The lease specifies the duration for which the Quoter service is available Once the lease expires, the Lookup service automatically removes the Quoter service reference Server containing Quoter service is resource user & Lookup service is resource provider When the Quoter service is registered at the Lookup service, a lease is created

49. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 49 Leasing – Known Uses Jini Jini couples each service with a lease object that specifies the duration of time for which a client can use that service Jini also associates a lease object with each registration of a service with the Jini lookup service. If a lease expires & the corresponding service does not renew the lease, the service is removed from the lookup service.NET Remoting Remote objects in.NET Remoting are managed by the Leasing Distributed Garbage Collector Dynamic Host Configuration Protocol (DHCP) A DHCP lease is the time that the DHCP server grants permission to the DHCP client to use a particular IP address Web-based e-mail accounts Many web-based e-mail accounts, for example MSN Hotmail or GMX become inactive automatically if not used for a certain period of time The time duration can be regarded as a lease whose renewal requires use of the e- mail account

50. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 50 Evictor – Example A three-tier Application with large number of business objects State of objects is persistent & so objects can be re-created Objects need to be in memory to respond to client request quickly However, keeping too many objects in memory can be expensive How do we balance the conflicting requirements of keeping objects in memory for efficiency & at the same time reducing their number to minimize performance degradation? NE Shelf Card

51. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 51 Evictor Context Systems that need efficient management of resources Problem How can the lifecycle of a resource be controlled by the frequency of use of the resource? How can release of resources be controlled by factors such as type of resource, available memory & CPU load? EvictorResource * 1 EvictionInterface isEvictable() info() beforeEviction Strategy Solution Evictor monitors the use of a resource It then controls its lifecycle using one or more strategies such as LRU Resources that are selected using the strategies are evicted

52. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 52 Evictor – Example Resolved NE, Shelf, & Card objects would implement EvictionInterface Each time an object is used, it would get marked with a timestamp Periodically, the Evictor would run & use a strategy such as LRU to identify objects that have not been marked Objects that are not marked would be evicted & their corresponding resources released Before being evicted, objects would be given a chance to do any cleanup NE Shelf Card Objects Selected

53. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 53 Evictor – Known Uses Enterprise Java Beans (EJB) Activation & deactivation mechanism is provided by the container to swap out beans from memory to secondary storage Java Reference API JDK 1.2 introduced the concept of reference objects Soft, weak & phantom references CORBA Servant Manager keeps track of number of instantiated servants It evicts servants depending upon memory requirements Paging OS copies a certain number of pages from the storage device into main memory. When a program needs a page that is not in main memory, the OS evicts a page from memory & copies it to the storage device It then copies the required page from the storage device into main memory

54. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 54 Resource Management Patterns Map Leasing Lazy Acquisition Evictor Partial Acquisition Eager Acquisition Pooling Resource Lifecycle Manager Caching CoordinatorLookup

55. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 55 Case Study: Mobile Network Overview System Architecture Hot Spot Analysis Base Station Operation & Maintenance Center Pattern Map

56. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 56 System Overview

57. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 57 Base Station Responsible for communicating with mobile phones Antennas are used to transmit digitized voice via radio signals Mediates calls between mobile phones & RNCs Operated & maintained by the OMC

58. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 58 Operation Maintenance Center (OMC) The Operation & Maintenance Center consists of User Interface Topological Tree Tree Management

59. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 59 System Interaction – Call Establishment

60. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 60 System Interaction – Call Hand-over

61. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 61 Base Station Hot Spot: Flexible Application Discovery Problem Application components are encapsulated & provide service interfaces In order to connect to each other, how can the applications find each other’s service interfaces? Solution Service interfaces are registered with a lookup service (Lookup) Use the lookup service (Lookup) to find service interfaces of used components Lookup Table Connection Mgmt Call Processing User Interface

62. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 62 Base Station Hot Spot: Concurrency Problem Many concurrent activities have to be performed, such as call establishment & performance management Multiple threads are created to guarantee responsiveness of the base station How can a number of threads that are created be controlled such that there is optimal resource utilization? Solution Use several threads, but limit the number of threads Reuse threads (Pooling)

63. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 63 Base Station Hot Spot: Connection Management Problem Connections are not fixed & are also unreliable How can system resources be freed on time when connections are broken? Solution Free resources associated with a connection automatically (Evictor)

64. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 64 Base Station Hot Spot: High Performance Problem Interactions between CPU boards are expensive The system must guarantee stringent Quality of Service properties How can unpredictable overhead be avoided from dynamic resource acquisitions, such as connection establishment & memory allocations? Solutions Avoid expensive resource acquisitions at run-time (Eager Acquisition) Reuse once acquired resources (Caching)

65. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 65 Base Station Hot Spot: Failure Handling Problem OMC service interface may become unavailable & bind resources unnecessarily Sudden disconnections cannot be avoided How can the base station ensure that a reference to a service interface that is no longer valid is not used? Solution Lease reference to OMC service interface (Leasing) Prolong leases regularly In case of failure a new OMC may contact the base station

66. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 66 Base Station Pattern Map

67. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 67 OMC Hot Spot: Station Discovery & Communication Problem Base stations have to be connected to the OMC Base station service interfaces are exposed via references How can the service interfaces be obtained, since the IP addresses of base stations are not enough to communicate with them? Solution Fetch references of application components using a special protocol Store those references and allow their retrieval (Lookup)

68. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 68 OMC Hot Spot: Network State Maintenance Problem Network connections are inherently unreliable How can the state of a network element be obtained if the network connection to that element is broken? Solution Cache state of individual network elements (Caching) Represent network elements via proxies Inform operator that visible state represents possibly outdated information RNC Base Station Base Station CPU Board Base Station Operation & Maintenance Call Processing

69. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 69 OMC Hot Spot: High Performance Problem User Interface needs to be highly responsive How can the OMC be made highly responsive to user input, such as when selecting network elements & viewing their properties? Solution Buffer once retrieved data (Caching) Caching helps to improve performance, avoiding expensive data acquisition

70. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 70 OMC Hot Spot: High Scalability Problem OMC manages thousands of network elements Network elements contain large state information How can the OMC be made scalable to cope with all the network elements it manages? Solution Evict node state information that is not needed any longer (Evictor) Acquire large state in steps, for example large alarm logs (Partial Acquisition)

71. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 71 OMC Hot Spot: Maintenance & Upgrade Problem Software updates need to be applied to network elements regularly How can it be ensured that all versions of the software are compatible & consistent? Solution Coordinate the download & installation (Coordinator) Separate installation from activation

72. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 72 OMC Hot Spot: Complexity Problem Managing resources explicitly is tedious Lifecycles might be non-trivial & depend on each other How can the lifecycles of interdependent resources be managed? Solution Introduce a Resource Lifecycle Manager to coordinate the lifecycle of complex resources Connection Thread Memory

73. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 73 OMC Pattern Map

74. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 74 Case Study: Ad Hoc Networking – Overview Ad hoc networking is based on the principle of spontaneous addition, discovery, & use of services in a network Services can be of multiple types, such as Simple time services PowerPoint presentation services MP3 player services All ad hoc networking technologies Are either platform- or programming language-dependent, But they share a common underlying architecture

75. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 75 Motivation Consider a mobile network as the ad hoc network Network consists of mobile devices containing software that accesses distributed services Services can include A stock quote service that delivers real time stock quotes A news service that delivers the latest news, customized according to user preferences An advertisement service that delivers advertisements according to user preferences & location Services themselves reside on the mobile phone & communicate with back-end providers to obtain necessary information However, devices such as mobile phones are typically constrained by memory, processing power, & storage, & therefore can not host several services simultaneously Mobile Phone Runtime Environment Application Software Dynamically loaded Service

76. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 76 Forces Efficiency Unnecessary service acquisitions should be avoided. System load caused by unused services must be minimized. Simplicity Management of services used by a client should be simple. It should be optional for a client to explicitly release services it no longer needs. Availability Services not used by a client, or no longer available, should be freed as soon as possible to make them available to new clients. Adaptation The frequency of use of a service should influence the lifecycle of a service. Service release should be determined by parameters such as available memory & CPU load. Actuality A device should not use an obsolete version of a service when a new version becomes available. Transparency The solution should be transparent to clients. The solution should incur minimum execution overhead & software development complexity.

77. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 77 Solution Apply the resource management pattern language The following are the four main components that are responsible for the key interactions in applying the resource management pattern: Service – A service is published so that it can be discovered & used by clients in the ad hoc network Lookup service – A lookup service is used by service providers that register services, & by clients to discover the services Client – A client, such as a mobile device, is an entity that makes use of a service. The client finds a service using the lookup service Service provider – A service provider is an entity that provides a service for clients by registering the service, or only part of it, with the lookup service Service ProviderClient «uses» Service «uses»«provides» «uses» Lookup Service «uses»

78. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 78 Example: Mobile Phone – Solution Structure ServerMobile Phone Infrastructure Server Acquisition Interface Registraton/Lookup Interface Application Software : Client Stock Exchange : Service Provider Quoter Service : Service Directory : Lookup Service Quoter Proxy : Service Proxy Service Interface

79. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 79 Example: Mobile Phone Lazy Acquisition The user of the mobile phone can decide any time to use a service The services are loaded lazily – the service must not be loaded in advance The directory is consulted on demand for services to be downloaded on the mobile phone Server Mobile Phone Infrastructure Server Registration/Lookup Interface Application Software : Client Directory : Lookup Service ? ?

80. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 80 Example: Mobile Phone Lookup: Service Registration Stock exchange service provider acts as a resource provider, while the quoter service is the actual resource that it provides Stock exchange creates & registers stock quoter service with the directory Stock exchange service provider submits a reference to itself along with a set of properties that describe the services that it provides Stock exchange service provider registers quoter proxy with the lookup service

81. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 81 Example: Mobile Phone Lookup: Service Discovery Application software of mobile phone first finds the directory It then downloads the client proxy & installs it It finally acquires the quoter service & uses it to fetch stock quotes

82. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 82 Example: Mobile Phone Leasing, Evictor Once the mobile phone finds a service, it acquires a lease to gain access to the service A lease is a grant of guaranteed access for a specific time Each lease is negotiated between the client & the service provider References to service providers are automatically removed from the directory that no longer provide any services, & therefore whose leases have expired Corresponding resources are automatically evicted

83. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 83 Example: Mobile Phone– Pattern Map Service Provider Client Service Lookup Service Evictor Leasing

84. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 84 Summary The presented patterns Address non-functional properties, such as Scalability, Performance, Predictability, Stability, Consistency, & Flexibility Cover the complete lifecycle from acquisition to release Are independent of any application domain Are generalizations of more specialized patterns Not addressed topics in this tutorial Access to concurrent resources, see POSA2 & Doug Lea’s book “Concurrent Programming in Java” Fault tolerance techniques, to be published in 2007 Reference counting, see POSA 4/5 Extension of the pattern language with new experiences in Grid computing – sharing & aggregation of distributed resources Autonomic computing – automated resource management anticipating resource need

85. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 85 Other Literature The POSA3 patterns are tightly integrated with other pattern literature

86. Patterns for Effective Management of Resources in Software Systems Douglas C. Schmidt, Prashant Jain, & Michael Kircher 86 Pattern-Oriented Software Architecture, Volume 3: Patterns for Resource Management Michael Kircher, Prashant Jain John Wiley & Sons ISBN 0-470-84525-2 http://www.posa3.org