Bell Laboratories1 Organizational Patterns James O. Coplien Bell Laboratories Naperville, Illinois, USA JaCC Software Developers 16 September : :30

Bell Laboratories2 Organizational Patterns James O. Coplien Bell Laboratories Naperville, Illinois, USA IIT Colloquium Series Tuesday, February 8, 2000 M-50 Auditorium

Bell Laboratories3 Overview What are organizational patterns? Methodology Our model of organizational change Using patterns for change management The patterns themselves can be found at

Bell Laboratories4 What is a pattern? A solution to a problem in a context Architectural patterns ideas first published by Alexander in 1977 Look at issues of structure, not just parts Build on proven practice, not “promising theories” Adopted by the software community starting in 1990; started gaining critical mass by 1993 Today, widely used as a design tool, especially in the OO community

Bell Laboratories5 What are organizational patterns? Solutions to organizational problems in a context First appeared in the Alexander + software context at PLoP in 1994 (Coplien, Whitenack); received with some skepticism Now, a growing body of knowledge Or, a construct from anthropology, Kroeber: »Universal patterns: transcend cultures »Systemic patterns: have a common root in an ancient culture »Total culture patterns: give a culture its identity Patterns define culture

Bell Laboratories6 Why do we care about organizational patterns? Because process-based approaches have serious limitations that hark back to the era of the industrial revolution Because experience with project management, technology transfer, etc. show software to be a primarily social activity Because of the tie to cultures, and the study of cultures is about social activity Because we should care about human comfort in addition to product quality Because they work!

Bell Laboratories7 The Pasteur Process Research Project at Bell Labs Goal: To know how to design highly productive software development organizations Premise: Organization drives process, which drives productivity Strategy: Correlate properties of organizations with order-of-magnitude productivity Technique: Patterns

Bell Laboratories8 A Theory of Organizational Change The idea of an organization as an anthropomorphic entity »Some are intelligent: good reactions, learn from mistakes »Some are stupid: react in insane ways, don’t learn Organizations can learn! Organizations can’t be taught Therefore, learning is experiential »Experience is a hard teacher Can be helped with empirical foundations and introspection

Bell Laboratories9 Empirical foundations “Pattern Mining” Finding recurring structures and practices in healthy organization To a degree, seeking the absence of these structures in dysfunctional organizations A soft science Akin to anthropology and its many branches

Bell Laboratories10 Organizational Pattern Mining Based on social network analysis with informal extensions Organizational role-play to gather social network data Dyadic and triadic sociometric data Uses CRC cards Light facilitation Half-day exercise

Bell Laboratories11 CRC Cards: Classes, Responsibilities, and Collaborators Subsystem coord. Validate MR lists Build group products Administer ENVY Resolve logical deps. Subsystem coord. Change committee Designers System test

Bell Laboratories12 Work-life Role-Play Identify project roles Study subjects play roles Development scenarios drive role-play Capture interaction & coupling on CRC cards Social Network Analysis Tools »Organization Structure Visualization »Organizational Metrics Capture Trends as Generative Patterns

Bell Laboratories13 Distilling the Patterns Sociometric analyses »Pasteur analysis tool »Analytical studies of data Catalog sociograms, sociomatrices, sociometric data The pattern must recur The pattern must solve a problem Document in pattern form

Bell Laboratories14 QPW as a Subject Fourth generation product in one generation Remarkable press reviews The most effective organization we have found »Productivity »Quality »Interval »Market Used to calibrate our “spectrum” Analysis done at my request I am not a QPW user

Bell Laboratories17 Organization Metrics QPW

Bell Laboratories18 Writers’ Workshops A forum to review patterns Borrowed from the creative literature community Designed to protect the dignity of the author Several basic steps: »Moderator introduces author »Author reads the work »Author becomes a fly on the wall »Someone summarizes the pattern »The strengths are emphasized »Suggestions for improvement »Author asks for clarification »Thank the author

Bell Laboratories19 Using the Patterns Let dysfunctional organizations become exposed to the patterns to see what is possible Use organizational role-play as an introspection tool Collect and process sociometric data from the role play Present the data to the organization for a second round of introspection—a “mirror” Learning and growth build on the insights

Bell Laboratories20 The Organizational Role-Play Same technique as used for pattern mining Largely self-running Role-based »Useful in most development organizations Actor-based »Useful for diagnosing specialized team problems »“Ethnological” studies

Bell Laboratories21 Facilitation Guidelines Involve everybody Don’t try solving other peoples’ problems »Other organizations must solve their own problems »Interface issues can be dealt with in another forum »Interfaces generally involve politics Don’t look for “the problem” »Problems are systemic »Solutions are generative »Let the patterns cause the “aha”s

Bell Laboratories22 More principals of organizational change Happens bottom-up, perhaps with the top-down support of a sponsor or patron Patterns are closely tied to value systems—which must be elicited bottom-up Organizational change means building a new culture: reverse anthropology »New language »New normative behaviors »New values »New symbols »New stories (patterns) »New rituals

Bell Laboratories23 What makes organizations excel? No silver bullet Well-known productivity enablers A little luck The right balance of techniques to make a working social (and economic and technical and...) system

Bell Laboratories24 Communication is Key The “Buffalo Mountain” pattern Decentralized, yet a graceful distribution Almost fully connected Anti-schmisogenetic »No splinter groups »No time-serial sub-processes »No “subroutines”

Bell Laboratories25 Communication Intensity QPW

Bell Laboratories26 Size Easy to socialize the architecture Self-directing »Correlates to Gerry Weinberg’s SEI/CMM parallel »The true hallmark of a mature organization Divide and conquer Essentially large organizations need hierarchy

Bell Laboratories27 Architect Also Implements Problem: Preserving the architectural vision through to implementation Context: An organization of developers that needs strategic technical direction. Forces: Totalitarian control is viewed by most development teams as a draconian measure. The right information must flow through the right roles. The architect is one of the few people, or may be the only person, with the big picture. Big picture knowledge is important even during implementation, especially when using iterative development. But the architect usually isn’t around during implementation. Solution: Beyond advising and communicating with developers, architects should also participate in implementation. Resulting Context: A development organization that perceives buy-in from the guiding architects, and that can directly avail itself of architectural expertise. The architect learns from implementation constraints and may modify the architecture accordingly. Rationale: Vitruvius notes: “...[A]rchitects who have aimed at acquiring manual skill without scholarship have never been able to reach a position of authority to correspond to their pains, while those who relied only upon theories and scholarship were obviously hunting the shadow, not the substance. But those who have a thorough knowledge of both, like men armed at all points, have the sooner attained their object and carried authority with them.” Larry Constantine emphasizes the value of “street creds” in an architect.

Bell Laboratories30 Distribute Work Evenly Like QPW adjacency graph Sociometric analysis: low graph centrality Can be remedied by load leveling, staffing, splitting roles, combining roles

Bell Laboratories31 Organization Metrics for Distribute Work Evenly QPW

Bell Laboratories32 Personnel Issues Personal excellence and integrity Highly specialized Balance of teamwork (communication) and individuation (specialization, personal sign- offs) How does one assemble such a group? 1. Magic: A secret that only Philippe knows 2. Outside interests? (Gabriel) 3. Self-selecting Team (another pattern)

Bell Laboratories33 Engage Quality Assurance QA is tightly coupled in the social network Large beta program–customer engagement Developers reserve early verification to themselves QA first-in, last-out

Bell Laboratories34 Reward Excellence A sense of contribution »Reinforced in project signoffs »Reinforced in reward system Financial incentives »No first-hand insights into QPW »Controversial, but claims demonstrable results Two cultures with extravagant rewards: 1. The U. S. West Coast 2. Financial trading Overcomes fear of change

Bell Laboratories35 Process? The ability to introspect well The ability to self-manage Organizational structure over task sequencing Effective meetings, and lots of them

Bell Laboratories36 Conclusion Organizational patterns are: »Patterns that capture the “architecture” of organizations »In anthropology, the fabric of social structure Role-playing data gathering and introspection Organizational change is generative Organizations can use patterns as exemplars for improvement A rich body of literature exists and is growing