Agility, Improvisation, or Enacted Emergence? Dr Yingqin Zheng, Dr Will Venters, Dr Tony Cornford This research was undertaken as part of Pegasus EPSRC: Grant No: EP/D049954/ Dec 2007 ICIS, Montreal

Introduction  Pegasus Project – Exploring practices of GridPP: the UK particle physics Grid  Agile systems development?  Methodology as faked? Fiction? Amethodical?  Organisational improvisation  Improvisation Paradoxes  Enacted Emergence?

Organizational Improvisation  Metaphors  Jazz (Weick 1992, 1999; Barrett 1998, Hatch 1999)  Improvisational Theatre (Crossan, 1998)  Cunha (1999): “ the conception of action as it unfolds, by an organisation and/or its members drawing on available material, cognitive, affective and social resources”  Convergence in time of conception and execution  Bricolage – finding solutions from available rather than optimal resources

Improvisation Paradoxes Situated Improvisation environmental turbulence task uncertainty unplanned-for occurrences task complexity drop your tools visions (Moorman and Miner, 1998, Ciborra, 1996); (Dahlbom and Mathiassen, 1993) (Miner et al., 2001) (Hutchins, 1995, Weick and Roberts, 1993) (Weick, 1993a) (Hatch, 1999, Mintzberg and McHugh, 1985, Hutchins, 1991, Weick, 1993b) Structured Chaos organized anarchy Persistent structures collateral structure experimental culture aesthetic of imperfection a sense of urgency (Cohen et al., 1972) (Lanzara, 1999) (Cunha et al., 1999) (Weick, 1999) (Crossan, 1998, Hutchins, 1991,Mirvis,1998) Planned Agility convergence of planning & execution plan to improvise mixing the pre-composed & the spontaneous magnetic fields artful planning (Moorman and Miner, 1998) (Miner et al., 2001) (Weick, 1998) (Weick, 1993a) (Baskerville, 2006)

Improvisation-Paradoxes Cont. Reflective Spontaneity retrospective sense- making ex post interpretation transient constructs emergent order (Weick, 1993b) (Lanzara, 1999) (Miner et al 2001) Collective Individuality (Mirvis, 1998) facilitative leadership trust and kinship fluid communication influence and persuasion hanging out (Crossan, 1998) (Crossan, 1998, Weick, 1993a) (Orlikowski, 1996, Miner et al., 2001) (Hatch, 1999) (Barrett, 1998) Anxious Confidence (Mirvis, 1998) moods individual skills & creativity formative context organizational memory (Ciborra, 2002) (Hutchins, 1991, Moorman and Miner, 1998, Orlikowski, 1996) (Ciborra and Lanzara, 1994) (Moorman and Miner, 1998)

Particle Physicists and Grids  Currently constructing the worlds most powerful particle accelerator… the Large Hadron Collider (LHC)  Searching for Higgs Boson – “1 person in 1000 worlds, or a needle in 20 million haystacks”  million gigabytes per year.  100,000 CPUs.  40PB disk, 40PB tape.  “Worlds biggest Grid“ CD stack with 1 year LHC data (~ 20 km)

Background Context  Building the LHC Computing Grid (LCG):  Highly distributed, complex and poorly defined systems development task.  Cutting edge hardware and software used.  New software standards being negotiated.  Middleware and support software being developed in a range of languages.  Grid must be distributed and proceed at different paces because of funding.  Particle physics has a long tradition of such large scale global collaborations (Traweek 1988). GridPP (UK Contribution to LCG)  To a significant degree agile…  Collaboration of 230 people in 19 UK universities, RAL and CERN.  Decisions are made democratically and consensually, and implemented by influence and persuasion.  Network rather than hierarchy  Virtual, federated, overlapping and inter-connected.  Virtual meetings, wikis, blogs, mailinglists

Research Findings Situated Improvisation EGEE, LCG, e-science, funding, hardware, software… Structured Chaos No top down authority; extensive management structure/communicative channels; competing technical solutions Planned Agility “day to day we keep putting one foot in front of the other … and different people, depending on their role in the project, are more oriented towards the ultimate goal or more oriented towards the little concrete footsteps that need to be taken...” Reflective Spontaneity -pragmatic, “getting the job done”, fire-fighting -monitoring, accounting, sense-making Collective Individuality -freedom to improvise and innovate -shared goal, trust, facilitative leadership, “hanging out” Anxious Confidence-pressure from LHC switch on; “Yes it will work.” -history of cutting-edge computing and large collaborations

Enacted Emergence  Enactment (Weick 1977)  “people invent organizations and their environments and these inventions reside in ideas that participants have superimposed on any stream of experience (ibid. p. 196)”.  Emergence  Temporally emergent qualities  Interactions of existing elements  In a historical context  The evolutionary approach of system development (Dahlbom and Mathiassen 1993)  Enactment of sensemaking

Enacted Emergence Environment Complexity, uncertainties, visions, pressure, risks History organizational memory of improvisation, history of innovation, Chaos trial and error, improvisation, bricolage Order continuity, stability, resilience Individuals competent, confident, creative, committed, pragmatic Collective shared goal, trust, hanging out, emotional bond, facilitative leadership, aesthetic of imperfection Planning broad direction, retrospective sense- makingsensemaking Unfolding democratic debates, spontaneous actions, natural selection Practices tinkering, innovation, invention Structure collateral, de-layered, democratic, communicative

Contributions  Improvisation paradoxes  Agility should embody a deliberate or natural mixture of structure and improvisation, order and changes, intentionality and flexibility, spontaneity and reflexivity, collectivity and individuality  Agile systems development “in the wild”  Embeddedness of agility  Large group performance is possible when the ambience is right.  Science vs art  Enacted Emergence  Duality between structure and agency