Applied Architecture (or… Architecture In Action) David Woollard University of Southern California Software Architecture Group NASA Jet Propulsion Laboratory Data Management Group

Goals of This Lecture In this lecture, we will cover: – Terminology Review: Styles, Patterns, etc. – Example 1: REST & the WWW Understanding a system in terms of its architecture – Example 2: Metadata Repository & Data Curation Patterns, styles and architecture at work – Example 3: DSSA for Scientific Computing How can architect help domain practitioners – Lessons Moving Forward

Architecture is Pervasive All software systems have an architecture All software systems have an architect Architecture is NOT a single phase – Design != Architecture Architecture is a process and artifacts

Conceptual Design Tools Abstraction and Terminology – What are the fundamental concepts in your system? Separation of Concerns – Isolate likely change – e.g., components & connectors, object orientation Refined Experience – What have other architects found useful?

Applying These Design Tools In this lecture, we will cover three systems: – REST and the WWW – Metadata Repository and Curation System – SWSA: a DSSA for Scientific Systems These systems illustrate: – Architecture as the key to system understanding – Styles, patterns, and design working together – Architecture as an aid to building better systems

The WWW Copyright © Richard N. Taylor, Nenad Medvidovic, and Eric M. Dashofy. All rights reserved.

The WWW An alternative view… Copyright © Richard N. Taylor, Nenad Medvidovic, and Eric M. Dashofy. All rights reserved.

WWW’s Architecture The application is distributed (actually, decentralized) hypermedia Architecture of the Web is wholly separate from the code There is no single piece of code that implements the architecture. There are multiple pieces of code that implement the various components of the architecture. – E.g., different Web browsers Stylistic constraints of the Web’s architectural style are not apparent in the code – The effects of the constraints are evident in the Web One of the world’s most successful applications is only understood adequately from an architectural vantage point. Copyright © Richard N. Taylor, Nenad Medvidovic, and Eric M. Dashofy. All rights reserved.

REST Principles [RP1] The key abstraction of information is a resource, named by an URL. Any information that can be named can be a resource. [RP2] The representation of a resource is a sequence of bytes, plus representation metadata to describe those bytes. The particular form of the representation can be negotiated between REST components. [RP3] All interactions are context-free: each interaction contains all of the information necessary to understand the request, independent of any requests that may have preceded it. Copyright © Richard N. Taylor, Nenad Medvidovic, and Eric M. Dashofy. All rights reserved.

REST Principles (continued) [RP4] Components perform only a small set of well- defined methods on a resource producing a representation to capture the current or intended state of that resource and transfer that representation between components. These methods are global to the specific architectural instantiation of REST; for instance, all resources exposed via HTTP are expected to support each operation identically. Copyright © Richard N. Taylor, Nenad Medvidovic, and Eric M. Dashofy. All rights reserved.

REST Principles (continued) [RP5] Idempotent operations and representation metadata are encouraged in support of caching and representation reuse. [RP6] The presence of intermediaries is promoted. Filtering or redirection intermediaries may also use both the metadata and the representations within requests or responses to augment, restrict, or modify requests and responses in a manner that is transparent to both the user agent and the origin server. Copyright © Richard N. Taylor, Nenad Medvidovic, and Eric M. Dashofy. All rights reserved.

REST and the WWW REpresentational State Transfer Style of modern web architecture Architectural principles are a way of understanding the WWW

Metadata Curation Example Catalog and Archive Service (CAS) Established as a subproject of OODT ( at JPL in 1999http://oodt.jpl.nasa.gov Refactored 4 years ago to incorporate: – Factory-based extension points – A client-server model supported by XML-RPC – A persistence model independent of implementation

Metadata Curation System Two CAS Subsystems: – CAS-File Manager: A metadata repository that manages an archive of data and a catalog of metadata about that archived data – CAS-Curator Interface: Web-based UI for the File Manager that orchestrates third-party metadata extractors, navigates local and remote directories, and handles data/metadata ingest Example of Design Patterns + Architectural Patterns + Architectural Styles

Types of Experience (Program) Design Patterns Styles Architectural Patterns Domain-Specific Software Architectures Application Domain Knowledge Scope Shallow Deep Programming (language level) Application Structure System Structure

CAS-File Manager Service Requirements: – The File Manager shall persist metadata in a database – The File Manager shall be runnable “out of the box” Note: I’ve made these up for the purposes of this class;) Consideration: – The factory method allows the developer to specify an interface but to defer that actual instantiation to subclasses

Design Solution Factory pattern & Abstract Factory pattern used to create multiple persistence interfaces implementations specified at runtime Two implementations were created: – Database implementation that connected to an external database that would need to be configured separately – Apache Lucene implementation that uses a flat-file indexing engine to build a local store of metadata documents.

Factory Interfaces The AbstractCatalogFactory in an interface that defines the factory creation methods for any Catalog Factory. The CatalogFactory is an interface class that defines the methods of any catalog implementation. The DataStoreCatalogFactory class defines the concrete construction methods for creating a catalog that persists metadata to a database. The DataStoreCatalog class defined the implementation of the catalog that persists metadata to a database.

CAS-File Manager Service Requirements: – The File Manager shall be runnable as a service – The File Manager shall provide a JAVA-based API Design Solution: – Client-server architecture can be used to create: A File Manager Service (or Server) A client with API that interacts with the service – Use REST-full services to provide: ingest, data delivery metadata query data discovery, etc.

CAS-Curator Example Project Goals: – Develop a web-based application that allows a user to manage data and associated metadata, including: extracting metadata from existing products changing metadata for previously stored products – Language agnostic (front-end could be PHP, Javascript, etc.) Design Solution: Application of Three-tier Pattern – Web-based GUI – File Manager provides web service interface – File Manager persists metadata to an underlying database

Patterns: Three-tier Three tier systems are very common in business applications: – Front tier is traditionally focused on user interaction – Middle tier is business or application logic – Back tier addresses data access and persistence Front TierMiddle TierBack Tier Request Reply

CAS-Curator Example

Persistence Presentation Logic

DSSA for Scientific Computing The Scientific Workflow Software Architecture (SWSA) is: – A domain-specific software architecture – USC research project founded about 4 years ago Example of Architecture impacting domain practitioners (in this case, computational scientists)

Research Problem How can we, as software engineers and architects, support scientists in the data production and distribution activities associated with complex computational science experiments? Insights: – State of the practice scientific computing development methodologies lack support for engineering concerns – Scientists are not motivated to be good software engineers – Despite a wealth of computational resources (clusters, supercomputers, clouds) and software infrastructure (grids, workflow systems), scientists are not using them. 25 SWSA: A DSSA for Workflow-Based Science Data Systems - March 2nd, 2010

Domain-Specific Soft. Arch. A DSSA includes: – An architecture known to work well in a particular domain – Reference components – Methodologies for applying the architecture to particular problems.

Workflow Systems SWSA is a workflow system: Workflows are dynamic models, interpreted at run-time, that capture both control flow and data flow Models orchestration of multiple executables, level-processors, and/or scientific tools Can be parameterized and replayed

SWSA: A DSSA for Scientific Software

Standardized Access to Services Grid Services are accessed through invoking connectors. Services include: – Provenance Services – Data Discovery Services – Fault Registries You’ll learn more about this as you work on your Course Project.

Looking Forward In conclusion: – Architecture is necessary to understand a system It captures design decisions and principles – Architecture can be a fusion of design patterns, architectural styles and architectural patterns that form a complex system – Architecture can guide the builder of future like systems In real life, a system’s architecture is complex. Rarely will you find a textbook example that fully comprises the architecture of your system.