Presentation on theme: "10:1 attributed copies permitted ES/SDOE 678 Engineering of Agile Systems and Enterprises Fundamentals of Analysis,"— Presentation transcript:
10:1 email@example.com@stevens.edu, attributed copies permitted ES/SDOE 678 Engineering of Agile Systems and Enterprises Fundamentals of Analysis, Synthesis, and Performance ES/SDOE 678 Engineering of Agile Systems and Enterprises Fundamentals of Analysis, Synthesis, and Performance Session 10 – The Edge of Knowledge and Term Project Planning School of Systems and Enterprises Stevens Institute of Technology, USA
10:3 firstname.lastname@example.org@stevens.edu, attributed copies permitted EXERCISE 1) Read the Project Guideline Document 2) Start an MS Word document called: 678D0- V1.doc (D0 signifies deliverable zero) 3) In this document: a) Outline section headings of your project report per guidelines you read b) Declare your project intention (same as or different than D1) c) Transfer this document to the instructor now (email or memory stick) start thinking…later, in a short while Guest Speaker Next: Vijay KumarVijay Kumar
10:4 email@example.com@stevens.edu, attributed copies permitted SDOE 675SDOE 678SDOE 679SDOE 683 678 Engineering 679 Architecting 683 Designing 675 Thinking SDOE 683 – Self Organizing Agile Systems and Enterprises: Architectural Patterns Enabling Self-Organizing Systems of Systems Common patterns converge here in a study of class 2 (reconfiguring) agile systems across a seemingly wide variety of interesting system types, characterized by aspects of complex adaptive systems expressed as self-organized systems of systems. Studies explore 4 th generation warfare, swarm systems, systems of systems with willful components, resiliency and vulnerability in infrastructure networks, emergent behavior, interoperability, open-community systems, attractors and generating functions as behavior boundaries, evolutionary systems, and issues at the moving edge of agile system and enterprise knowledge.
System of Subsystems System of Systems Lessons from the Channel Tunnel. Allen Fairbairn, Systems Engineer and Manager – Channel Tunnel Project Difference in Relationships
Analytical Analytical Reductionist Reductionist Deterministic Behavior Deterministic Behavior Controlled Environment Controlled Environment Predictable within defined limits Predictable within defined limits Critical Failure Modes Critical Failure Modes Relational Relational Holistic Holistic Emergent Behavior Emergent Behavior Dynamic Interactive Environment Dynamic Interactive Environment Essentially Unpredictable Essentially Unpredictable Degradable Failure Modes Degradable Failure Modes System of SubsystemsSystem of Systems Lessons from the Channel Tunnel. Allen Fairbairn, Systems Engineer and Manager – Channel Tunnel Project Difference in Relationships
10:7 firstname.lastname@example.org@stevens.edu, attributed copies permitted Complex Adaptive Systems Self Organizing Systems Systems of Systems Swarm Systems Evolving Systems...and more 683 - Self Organizing Systems of Systems Sense-making…Looking for common patterns in… Autonomous Agent Systems Open Community Systems Network Systems Willful Systems UPAN Systems HIT Systems Resilient Systems Robotic Systems
10:8 email@example.com@stevens.edu, attributed copies permitted Overarching Themes for Air Force S&T United States Air Force Chief Scientist (AF/ST), 15 May 2010, Report on Technology Horizons, A Vision for Air Force Science & Technology During 2010-2030, Volume 1, AF/ST-TR-10-01-PR. www.af.mil/shared/media/document/AFD-100727-053.pdf www.af.mil/shared/media/document/AFD-100727-053.pdf The strategic context and enduring realities identified in “Technology Horizons” lead to a set of 12 “Overarching Themes” to vector S&T in directions that can maximize capability superiority. These shifts in research emphases should be applied judiciously to guide each research area. 1.From … PlatformsTo … Capabilities 2.From … MannedTo … Remote-piloted 3. From … FixedTo … Agile 4. From … ControlTo … Autonomy 5. From … IntegratedTo … Fractionated 6. From … PreplannedTo … Composable 7. From … Single-domainTo … Cross-domain 8. From … PermissiveTo … Contested 9. From … SensorTo … Information 10. From … OperationsTo … Dissuasion/Deterrence 11. From … Cyber defenseTo … Cyber resilience 12. From … Long system lifeTo … Faster refresh
10:9 firstname.lastname@example.org@stevens.edu, attributed copies permitted With SmartBird, Festo deciphers the flight of birds www.festo.com/cms/en_corp/11369.htm Inspired by the herring gull. SmartBird can start, fly, and land autonomously. Its wings not only beat up and down, but also twist at specific angles. File1.75 File1
10:10 email@example.com@stevens.edu, attributed copies permitted From MIT Open Course Ware: 16.410 and 16.412: Principles of Autonomy and Decision Making Prof Brian Williams, Prof Emilio Frazzoli and Sertac Karaman September, 8th, 2010
10:11 firstname.lastname@example.org@stevens.edu, attributed copies permitted jpg image “Accidents are entirely plausible and have happened elsewhere: In September 2011, an RQ-Shadow UAV crashed into a military cargo plane in Afghanistan, forcing an emergency landing. Last summer, test-flight operators of a MQ-8B Fire Scout helicopter UAV lost control of the drone for about half an hour, which traveled for over 20 miles towards restricted airspace over Washington DC. A few years ago in South Africa, a robotic cannon went haywire and killed 9 friendly soldiers and wounded 14 more. “Errors and accidents happen all the time with our technologies, so it would be naïve to think that anything as complex as a robot would be immune to these problems. Further, a robot with a certain degree of autonomy may raise questions of who (or what) is responsible for harm caused by the robot, either accidental or intentional: could it be the robot itself, or its operator, or the programmer? Will manufacturers insist on a release of liability, like the EULA or end-user licensing agreements we agree to when we use software--or should we insist that those products should be thoroughly tested and proven safe? (Imagine if buying a car required signing a EULA that covers a car's mechanical or digital malfunctions.) Drone-Ethics Briefing: What a Leading Robot Expert Told the CIA 15 Dec 2011, Patrick Lin, www.theatlantic.com/technology/archive/2011/12/drone-ethics-briefing-what-a-leading-robot-expert-told-the-cia/250060/
10:12 email@example.com@stevens.edu, attributed copies permitted ~$70,000 Dec2011 www.gatewing.com A Belgium Company File4.5
10:13 firstname.lastname@example.org@stevens.edu, attributed copies permitted jpg image $10,600 28Oct2010 A Swiss Company 20Dec2011 – Swiss scientists at EPFL have taken drone planes called swinglets built by a start-up called senseFly and are programming them to flock like birdsflock like birds File2.75
10:14 email@example.com@stevens.edu, attributed copies permitted What Are We Doing? Questions we seek answers for: What makes SO-SoSes work (achieve, grow, behave properly, evolve, …)? What recurring patterns are seen in various kinds of successful SO-SoSes What are the metrics of SO-SoS success? What are useful SO-SoSes to observe and analyze for clues? What knowledge can be utilized now – where and how? What information can be tested now – where and how? What data can be experimented with now – where and how? We are developing a “pattern language” for discourse – we can’t talk about concepts or think about them if we have no words for them, nor assemble them into meaningful constructions if we don’t have a grammar.
10:15 firstname.lastname@example.org@stevens.edu, attributed copies permitted System Security is a Prime SO-SoS Learning Opportunity Observed Asymmetric Advantages of the Artificial-System Adversary Adversary leads with innovation and evolution Adversary is a natural system, current security strategy is an artificial system Adversary self-organizes as a dynamic system-of-systems Architecture: Multi-agent Loosely coupled Self organizing Systems-of-systems Behavior: Swarm intelligence Tight learning loops Fast evolution Dedicated intent Assumptions: All systems are prey. The goal of a “natural” SO-SoS is survival. Fundamental natural strategies for survival are innovation and evolution. Currently the artificial-system predator has superior “natural” strategies. Natural systems have evolved very successful survival patterns. Artificial-system predators have evolved very successful attack patterns. The best Test & Evaluation is confrontation with the intelligent adversary!
10:16 email@example.com@stevens.edu, attributed copies permitted Its not about Cyber Security …all systems are prey Its about co-evolving self-organizing systems of systems, each with first priority on securing and maintaining existence. Maslow’s Hierarchy of Needs (for systems that would live one more day) 1 st Order: Core necessity 2 nd Order: As affordable Maslow’s Hierarchy of Needs (5) Discretionary: non-functional performance of existence (community impact) (4) Quality: functional performance of existence (3) Functionality: product of existence (reason for, purpose of) (2) Security: sustains existence (1) Energy: enables existence
10:17 firstname.lastname@example.org@stevens.edu, attributed copies permitted Maslow’s Hierarchy of Needs (for systems that would live one more day) 1 st Order: Core necessity 2 nd Order: As affordable Maslow’s Hierarchy of Needs Energy Needs Security Needs Functionality Performance Harmony Its not about Cyber Security …all systems are prey Its about co-evolving self-organizing systems of systems, each with first priority on securing and maintaining existence. (5) Discretionary: non-functional performance of existence (community impact) (4) Quality: functional performance of existence (3) Functionality: product of existence (reason for, purpose of) (2) Security: sustains existence (1) Energy: enables existence
10:18 email@example.com@stevens.edu, attributed copies permitted Reality SO-SoS scares people - but SO-SoS are all around us - and the adversary thrives on it SysEs, SecEs and Decision Makers don’t communicate Only SysEs can enable next gen SecE: SO-SoS We need a common language and vision = OBJECTIVE - for SysEs, SecEs, and Decision Makers Patterns reflected from common understandings - solve communication problem - solve scary problem - brings shared vision into focus (Should you care to accept the mission….) You can be in the vanguard of SO-SoS pattern discovery - choose patterns useful to your work & knowledge dev. - suggested pattern concepts can be provided - source reference material can be provided - collaboration will be provided
10:19 firstname.lastname@example.org@stevens.edu, attributed copies permitted common language concepts comfort Systems Engineer Decision Maker Objective Met with Stories, Graphics, Metaphors, References Security Engineer
10:20 email@example.com@stevens.edu, attributed copies permitted Agile system security, as a minimum, must mirror the agile characteristics exhibited by the system attack community: [S]Self-organizing – with humans embedded in the loop, or with systemic mechanisms. [A]Adapting to unpredictable situations – with reconfigurable, readily employed resources. [R]Reactively resilient – able to continue, perhaps with reduced functionality, while recovering. [E]Evolving in concert with a changing environment – driven by vigilant awareness and fitness evaluation. [P]Proactively innovative – acting preemptively, perhaps unpredictably, to gain advantage. [H]Harmonious with system purpose – aiding rather than degrading system and user productivity. To Start: Mirror the Enemy www.parshift.com/Files/PsiDocs/Pap100226-AgileSecuritySelfOrganizingCoEvolution-ExtAbst.pdf
10:21 firstname.lastname@example.org@stevens.edu, attributed copies permitted Natural systems exhibit all six characteristics. Artificial self-organizing agile systems will have at least one combination that traces a path from S to H, for a minimum of four characteristics. S means the system dances at the pace set by situational reality. But by itself, if S provides no value (beat is right but dance is independent of situation), it is useless. If we have S and H, without value (benign result), it is still useless. A good example of S-A-R-H is exhibited by the New York subway control room. S w/o A doesn’t ensure things happen when they must/should. Axiom*: SAREPH Minimum Combinations Minimum = S & (A|E) & (R|P) & H Reactive Resilience Evolving Strategy Adaptive Tactics Self Organization Harmonious Operation Proactive Innovative SA PH E R Trace any/all paths from S to H * subject to change
10:22 email@example.com@stevens.edu, attributed copies permitted When a room has a window with a view, it is a focal point: people are attracted to the window and want to look through it. The furniture in the room creates a second focal point: everyone is attracted toward the point the furniture aims them at (the center of the room or a TV). This makes people feel uncomfortable. They want to look out the window, and toward the other focus at the same time. Rearrange the furniture so its focal point becomes the window, and everyone is comfortable. That's a very simple example, and there are literally hundreds more in this book and its sequel. The book's main idea is much more powerful than that. It applies to almost every aspect of life, not just to architecture. When a situation makes us unhappy, it is usually because we have two conflicting goals, and we aren't balancing them properly. Alexander's idea is to identify those ``conflicting forces'', and then find a solution which brings them into harmony. [Leonard R Budney, Amazon Reviewer] This four-volume work is Christopher Alexander's magnum opus of architectural philosophy, and a book on which he has been working for over twenty years. The essence of that view is this: the universe is not made of "things," but of patterns, of complex, interactive geometries. Furthermore, this way of understanding the world can unlock marvelous secrets of nature, and perhaps even make possible a renaissance of human-scale design and technology. [Michael Mehaffy, Amazon Reviewer] (read this one)(253 patterns)
10:23 firstname.lastname@example.org@stevens.edu, attributed copies permitted Alexander’s Pattern Form From: Alexander, Christopher. 1977. A Pattern Language. New York: Oxford University Press. First, there is a picture, which shows an archetypal example of that pattern. Second, after the picture, each pattern has an introductory paragraph, which sets the context for the pattern by explaining how it helps to complete certain larger patterns. Then there are three diamonds to mark the beginning of the problem. After the diamonds there is a headline, in bold type. This headline gives the essence of the problem in one or two sentences. After the headline comes the body of the problem. This is the longest section. It describes the empirical background of the pattern, the evidence for its validity, the range of different ways the pattern can be manifested in a building, and so on. Then, again in bold type, like the headline, is the solution—the heart of the pattern—which describes the field of physical and social relationships which are required to solve the stated problem, in the stated context. This solution is always stated in the form of an instruction— so that you know exactly what you need to do, to build the pattern. Then, after the solution, there is a diagram, which shows the solution in the form of a diagram, with labels to indicate its main components. After the diagram, another three diamonds, to show that the main body of the pattern is finished. And finally, after the diamonds there is a paragraph which ties the pattern to all those smaller patterns in the language, which are needed to complete the pattern, to embellish it, to fill it out. There are two essential purposes behind this format. First, to present each pattern connected to other patterns, so that you grasp the collection of... patterns as a whole, as a language within which you can create an infinite variety of combinations. Second, to present the problem and solution of each pattern in such a way that you can judge it for yourself, and modify it, without losing the essence that is central to it.
10:24 email@example.com@stevens.edu, attributed copies permitted Our Pattern Form Name:Descriptive name for the pattern. Context:Situation that the pattern applies to. Problem:Description of the problem. Forces:Tradeoffs, value contradictions, constraints, key dynamics of tension & balance. Solution:Description of the solution. Graphic:A depiction of response dynamics. Examples:Referenced cases where the pattern is employed. Agility:Evidence of SAREPH characteristics that qualify the pattern as agile. References:Literature access to examples. www.parshift.com/Files/PsiDocs/Pap100317Cser-OnDiscoveryAndDisplayOfAgileSecurityPatterns.pdf
10:25 firstname.lastname@example.org@stevens.edu, attributed copies permitted www.parshift.com/Files/PsiDocs/Pap100317Cser-OnDiscoveryAndDisplayOfAgileSecurityPatterns.pdf Dove, Rick and Laura Shirey. On Discovery and Display of Agile Security Patterns. 2010. 8th Conference on Systems Engineering Research March 17-19, Hoboken, NJ. www.parshift.com/Files/PsiDocs/Pap100317Cser- OnDiscoveryAndDisplayOfAgileSecurityPatterns.pdf Example of a pattern description synopsis. These descriptions are for path-finder patterns rather than well-known common-practice patterns, full understanding is either obtained from reading the referenced papers or from reading accompanying discussion pages. Note: the 4 rows for context-problem-forces- solution should be generic abstractions; the 2 rows for graphic panels and SAREPH agility should preferably ground the abstractions in a specific example. (not like this one: abstract everywhere)
10:26 email@example.com@stevens.edu, attributed copies permitted Aggressive shield waxes and wanes measure-for-measure in real time Example: Artificial immune system – detection, selection, cloning and retirement applied to mobile network intrusion detection and repulsion. See (Cheng et al. 2008, Edge et al. 2006). Example: Botnet denial of service defense – Instantly recruit an unbounded network of computers to shield a server from being overwhelmed by botnets. See (Dixon et al. 2008, Mahimkar et al. 2007). Example: Just-in-time drone swarms – Load disposable drones with modular sensor and weapon choices, and deploy quantities as needed. See SWARM, JITSA discussion in (Hambling 2006). Example: Plants – Use volatile signaling compounds to fend off attack, activate neighbor plants to do the same, and call in predators. See (Wilkinson, 2001). Above are systemically self-organized – here are some human directed examples NATO Internet Storm Center Fire department mutual aid Incident response coalitions (Khurana 2009) Dynamic Phalanx Defense
10:27 firstname.lastname@example.org@stevens.edu, attributed copies permitted Pattern: Horizontal Meme Transfer Name: Horizontal Meme Transfer (adapting patterns from other domains) Context: Systemic innovation and evolution. Problem:A need for improved system survivability, either reactive, proactive, or both. Forces:Evolution of innovation vs. evolution of robustness. Solution:Find relevant patterns in other domains and adapt them to the perceived threats and opportunities of the system of interest. Massive shared generation of intrusion detectors for evolving resilient-network vigilance circa 2010 From: Pattern Qualifications and Examples of next Generation Agile System-Security Strategies. www.parshift.com/Files/PsiDocs/PatternQualificationsForAgileSecurity.pdfwww.parshift.com/Files/PsiDocs/PatternQualificationsForAgileSecurity.pdf
10:28 email@example.com@stevens.edu, attributed copies permitted Pattern: Horizontal Gene/Meme Transfer Context: When conditions deteriorate, it makes a lot of sense to try to scavenge DNA from your neighbors. Horizontal gene transfer facilitates a fast microbial adaptation to stress. Higher-than-suspected transfer rates among microbes living in nutrient-poor environments, where sharing genes may be key to survival, has been observed. Evidence indicates that organisms limit gene exchange to microbes on nearby branches of the family tree, probably because their chromosomes share certain characteristics. Genes appear to be exchanged between species with similar chromosomal structures (Pennise 2011). Problem: Situational or environmental changes that threaten fitness or survival of the organism. Forces: Short-term adaptability vs. long-term-evolvability, horizontal gene transfer speeds the development of new traits by a factor of 10,000 (Woese 2000, Pennise 2011). Solution: Incorporate appropriate genetic material from other organisms that have developed compatible and useful situational fitness. Mobile genes don’t just help a community survive, they also provide the grist for evolutionary innovations. Horizontal gene transfer speeds up innovative short-term adaptation and long-term evolution Two modular gene pools Innovative adaptation and evolution Available high variety cellular organisms Intrachromsomal genes Extrachromosomal genes Rules 1.Packaging 2.Transfer 3.Entry 4.Establishment 5.Inheritance circa 2011 (Dove, Rick. 2011. Webinar: Toward a Systemic Will to Live –Patterns of Self-Organizing Agile Security. www.parshift.com/Files/PsiDocs/PatternQualificationsForAgileSecurity.pdf )
10:29 firstname.lastname@example.org@stevens.edu, attributed copies permitted Pattern: Horizontal Meme Transfer Examples: Horizontal gene transfer and evolution. (Woese 2000) & (Smets 2005). Cross-domain user-behavior-channeling pattern catalog. (Lockton 2009, 2010) Cross-domain dynamic-system process-pattern project. (Troncale 1978, 2006) Universal patterns in human activity and insurgent events. (Bohorquez 2009). Patterns in behavioral ecology and anti-predator behavior. (Blumstein 2010). Tradeoff between robustness and fragility in evolving complex systems. [S]elf organization controls the assembly process. [A]daptation occurs in assemblies that meet needs. [R]eactive resilience occurs with sufficient module mix to meet specific needs. [E]volution occurs in module and protocol upgrades. [P]roactive innovation occurs with speculative assemblies for unknown needs. [H]armony is maintained with a Highly Optimized Tolerance (Carlson 2002) small module and protocol repertoire in the knot. References: (see reference section, only URLs shown here. All accessed 1Jan2011)) (Blumstein 2010) www.eeb.ucla.edu/Faculty/Blumstein/pdf%20reprints/Blumstein_2010_BE.pdfwww.eeb.ucla.edu/Faculty/Blumstein/pdf%20reprints/Blumstein_2010_BE.pdf (Bohorquez 2009) www.nature.com/nature/journal/v462/n7275/full/nature08631.htmlwww.nature.com/nature/journal/v462/n7275/full/nature08631.html (Carlson and Doyle 2000) www.pnas.org/content/99/suppl.1/2538.full.pdf+htmlwww.pnas.org/content/99/suppl.1/2538.full.pdf+html (Lockton 2009) http://bura.brunel.ac.uk/bitstream/2438/3664/1/Lockton_SI_paper_disclaimer_added.pdfhttp://bura.brunel.ac.uk/bitstream/2438/3664/1/Lockton_SI_paper_disclaimer_added.pdf (Lockton 2010) http://danlockton.com/dwi/Download_the_cardshttp://danlockton.com/dwi/Download_the_cards (Smets 2005) www.nature.com/nrmicro/journal/v3/n9/pdf/nrmicro1253.pdfwww.nature.com/nrmicro/journal/v3/n9/pdf/nrmicro1253.pdf (Troncale 1978) www.allbookstores.com/author/International_Conference_On_Applied_General_Systems_Research_State_Uni.html www.allbookstores.com/author/International_Conference_On_Applied_General_Systems_Research_State_Uni.html (Troncale 2006) http://www3.interscience.wiley.com/journal/112635373/abstract?CRETRY=1&SRETRY=0http://www3.interscience.wiley.com/journal/112635373/abstract?CRETRY=1&SRETRY=0 (Woese 2000) www.ncbi.nlm.nih.gov/pmc/articles/PMC26958/pdf/pq008392.pdfwww.ncbi.nlm.nih.gov/pmc/articles/PMC26958/pdf/pq008392.pdf From: Pattern Qualifications and Examples of next Generation Agile System-Security Strategies. www.parshift.com/Files/PsiDocs/PatternQualificationsForAgileSecurity.pdfwww.parshift.com/Files/PsiDocs/PatternQualificationsForAgileSecurity.pdf
10:30 email@example.com@stevens.edu, attributed copies permitted Pattern: Bow Tie Processor (assembler/generator/mediator) Context: Complex system with many diverse inputs and many diverse outputs, where outputs need to respond to many needs or innovate for many or unknown opportunities, and it is not practical to build unique one-to-one connections between inputs and outputs. Appropriate examples include common financial currencies that mediate between producers and consumers, the adaptable biological immune system that produces proactive infection detectors from a wealth of genetic material, and the Internet protocol stack that connects diverse message sources to diverse message sinks. Problem: Too many connection possibilities between available inputs and useful outputs to build unique robust, evolving satisfaction-processes between each. Forces: Large knot short-term-flexibility vs small knot short-term-controllability and long- term-evolvability (Csete 2004); robustness to known vs fragility to unknown (Carlson 2002). Solution: Construct relatively small “knot” of fixed modules from selected inputs, that can be assembled into outputs as needed according to a fixed protocol. A proactive example is the adaptable immune system that constructs large quantities of random detectors (antigens) for unknown attacks and infections. A reactive example is a manufacturing line that constructs products for customers demanding custom capabilities. Millions of random infection detectors generated continuously by fixed rules and modules in the “knot” Evolve three fixed V-D-J gene-segment libraries Fixed-rule VDJ assembly with random interconnects Random high variety output with VDJ + VJ assemblies Available high variety genetic DNA input V: 123 Variable segments D: 27 Diverse segments J: 6 Joining segments increases to ~10 9 varieties with addition of random nucleotide connections between VDJ & VJ joinings ~10 6 VDJ+VJ possible antigen detector shapes V1V1 D1D1 VnVn VrVr JrJr DrDr rr 123 Vs 27 Ds 6 Js 1 random from each + random connect DnDn JnJn J1J1
10:31 firstname.lastname@example.org@stevens.edu, attributed copies permitted Example: Immune system--Millions of random infection detectors are generated continuously by fixed rules and modules Example: For immune system assembly process (Wikipedia 2010). For numbers (Li 2004). Example: Bow tie architecture for detector generation and sense-making. (Dove 2010). Example: Bow tie architecture for robust complex networks of many kinds. (Csete 2004). Example: General bow tie architecture and flexible-standards generation. (Hartzog 2010). [S]elf organization controls the assembly process. [A]daptation occurs in assemblies that meet needs. [R]eactive resilience occurs with sufficient module mix to meet specific needs. [E]volution occurs in module and protocol upgrades. [P]roactive innovation occurs with speculative assemblies for unknown needs. [H]armony is maintained with a Highly Optimized Tolerance (Carlson 2002) small module and protocol repertoire in the knot. References: (see reference section, only URLs shown here. All accessed 1Jan2011) (Carlson 2002) http://gabriel.physics.ucsb.edu/~complex/pubs/hot2.pdfhttp://gabriel.physics.ucsb.edu/~complex/pubs/hot2.pdf (Csete 2004) http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.173.3019&rep=rep1&type=pdf (Dove 2011) www.parshift.com/s/110411PatternsForSORNS.pdfwww.parshift.com/s/110411PatternsForSORNS.pdf (Hartzog 2010) http://blog.p2pfoundation.net/how-different-is-your-bow-tie/2010/06/21http://blog.p2pfoundation.net/how-different-is-your-bow-tie/2010/06/21 (Li 2004) http://bloodjournal.hematologylibrary.org/cgi/reprint/103/12/4602.pdfhttp://bloodjournal.hematologylibrary.org/cgi/reprint/103/12/4602.pdf (Wikipedia 2011) http://en.wikipedia.org/wiki/V(D)J_recombinationhttp://en.wikipedia.org/wiki/V(D)J_recombination Pattern: Bow Tie Processor (assembler/generator/mediator) From: Pattern Qualifications and Examples of next Generation Agile System-Security Strategies. www.parshift.com/Files/PsiDocs/PatternQualificationsForAgileSecurity.pdfwww.parshift.com/Files/PsiDocs/PatternQualificationsForAgileSecurity.pdf
10:32 email@example.com@stevens.edu, attributed copies permitted Pattern: Drag-and-Drop Framework and Modules Example: Adaptable Immune System Bow-Tie Antigen-Detector Generator detector sequence n short chain long chain detector sequence n+1 short chain long chain detector sequence n+2 short chain long chain 123 V segments 6 J segments27 D segments random nucleotides Infrastructure evolution Detector assembly Module pools and mix evolution Module inventory condition Combine two assemblies Add random nucleotides Use one each V-D-J Use one each V-J Infrastructure Modules Assembly Rules Integrity Management Active Passive genetic evolution bone marrow and thymus genetic evolution ??repair mechanisms?? cell Y detector antibody B-Cell V--D--J V--J From: Pattern Qualifications and Examples of next Generation Agile System-Security Strategies. www.parshift.com/Files/PsiDocs/PatternQualificationsForAgileSecurity.pdfwww.parshift.com/Files/PsiDocs/PatternQualificationsForAgileSecurity.pdf
10:33 firstname.lastname@example.org@stevens.edu, attributed copies permitted Context: A complex system or system-of-systems subject to attack and infection, with low tolerance for attack success and no tolerance for catastrophic infection success; with resilient remedial action capability when infection is detected. Appropriate examples include biological organisms, and cyber networks for military tactical operations, national critical infrastructure, and commercial economic competition. Problem: Directed attack and infection types that constantly evolve in new innovative ways to circumvent in-place attack and infection detectors. Forces: False positive tradeoffs with false negatives, system functionality vs functionality impairing detection measures, detectors for anything possible vs added costs of comprehensive detection, comprehensive detection of attack vs cost of false detection of self. Solution: A high fidelity model of biological immune system antibody (detection) processes that generate high quantity and variety of anticipatory speculative detectors in advance of attack and during infection, and evolve a growing memory of successful detectors specific to the nature of the system-of-interest. Speculative generation and mutation of detectors recognizes new attacks like a biological immune system Pattern: Proactive Anomaly Search
10:34 email@example.com@stevens.edu, attributed copies permitted Example: Lucid overview of antibody processes, including generation of speculative antibodies. See (Wikipedia 2010). Example: Artificial immune system general model applicable to cyber networks. See (Hofmeyr 2000). Example: Determining and evolving self and non-self behaviors in system call monitoring. See (Forrest 2008). Example: Detector cloning and mutation improvement. See (Hightower 1996). [S]elf organization occurs in negative selection, in limited-life positive selection, in deployment cloning, and in memory of the fittest detectors. [A]daptation occurs in bow-tie antibody (detector) creation, in negative selection and in positive selection. [R]eactive resilience occurs in constant refresh and replacement of useless and aged detectors. [E]volution occurs as the memory of effective detectors grows with exposure to attacks and infections. [P]roactive innovation is the process of the bow-tie speculative antibody creation. [H]armony is maintained by negative selection, and by limited-life purging of ineffective and of no-longer needed detectors. References: (see reference section, only URLs shown here. All accessed 1Jan2011) (Forrest 2008) http://www.cs.unm.edu/~forrest/publications/acsac08.pdfhttp://www.cs.unm.edu/~forrest/publications/acsac08.pdf (Hightower 1996) http://cs.unm.edu/~forrest/publications/baldwin.pdfhttp://cs.unm.edu/~forrest/publications/baldwin.pdf (Hofmeyr 2000) http://cs.unm.edu/~forrest/publications/hofmeyr_forrest.pdfhttp://cs.unm.edu/~forrest/publications/hofmeyr_forrest.pdf (Wikipedia 2011) http://en.wikipedia.org/wiki/Antibodyhttp://en.wikipedia.org/wiki/Antibody Dove, Rick, Patterns of Self-Organizing Agile Security for Resilient Network Situational Awareness and Sense-Making. 2011. www.parshift.com/Files/PsiDocs/PatternsForResilientNetworks.pdf www.parshift.com/Files/PsiDocs/PatternsForResilientNetworks.pdf Pattern: Proactive Anomaly Search
10:35 firstname.lastname@example.org@stevens.edu, attributed copies permitted Context: A decision maker in need of accurate situational awareness in a critical dynamic environment. Examples include a network system administrator in monitoring mode and under attack, a military tactical commander in battle, and the NASA launch control room. Problem: A very large amount of low-level noisy sensory data overwhelms attempts to examine and conclude what relevance may be present, most especially if time is important or if sensory data is dynamic. Forces: amount of data to be examined vs time to reach a conclusion, number of ways data can be combined vs number of conclusions data can indicate, static sensory data vs dynamic sensory data, noise tolerated in sensory data vs cost of low noise sensory data. Solution: Using a bow-tie process, each level looks for a specific finite set of data patterns among the infinite possibilities of its input combinations, aggregating its input data into specific chunks of information. These chunks are fed-forward to the next higher level, that treats them in turn as data further aggregated into higher forms of information chunks. Through feedback, a higher level may bias a lower level to favor certain chunks over others, predicting what is expected now or next according to an emerging pattern at the higher level. Each level is only interested in a small number of an infinite set of data-combination possibilities, but as aggregation proceeds through multiple levels, complex data abstractions and recognitions are enabled. Four level feed forward/backward sense-making hierarchy modeled on visual cortex Pattern: Hierarchical Sensemaking
10:36 email@example.com@stevens.edu, attributed copies permitted Example:Cortical Spatial Sensing – Visual cortex receives noisy retinal raster of ~1,000,000 points and recognizes prior learned patterns in the field of view. See (Serre 2007). Example:Cortical Temporal Sensing – Cortex receives time sequenced sensory input and constantly predicts what is expected next according to prior learned patterns. See (George 2009). Example:Network Anomaly Sensing – Level 1 network agents detect anomalies on hosts, Level 2 agents interpret Level 1 alerts and cause inter-host collaboration, Level 3 agents set policy for Level 2 and interface with humans at Level 4, Level 4 is human decider on action and advisor to Level 3. (See Haack 2009). [S]elf organizing sense-making emerges from feed forward/backward interplay resolution path through the four levels. [A]dapts to noisy input with suggested clean-up. (Learning evolves the content of levels, but is not part of this pattern). [P]roactive prediction of next temporal input feeds back expectations/suggestions to lower levels. [H]armony is maintained with decision making levels receiving situational awareness as succinct and relevant information appropriate with processing capability. References: (see reference section, only URL shown here, all accessed 1Jan2011) (George 2008) www.numenta.com/htm-overview/education/DileepThesis.pdfwww.numenta.com/htm-overview/education/DileepThesis.pdf (Haack 2009) www.cs.wfu.edu/~fulp/Papers/mims09f.pdfwww.cs.wfu.edu/~fulp/Papers/mims09f.pdf (Serre 2007) http://cvcl.mit.edu/Papers/SerreOlivaPoggioPNAS07.pdfhttp://cvcl.mit.edu/Papers/SerreOlivaPoggioPNAS07.pdf Dove, Rick, Patterns of Self-Organizing Agile Security for Resilient Network Situational Awareness and Sense-Making. 2011. www.parshift.com/Files/PsiDocs/PatternsForResilientNetworks.pdf Pattern: Hierarchical Sensemaking
10:37 firstname.lastname@example.org@stevens.edu, attributed copies permitted Blumstein, Daniel T. 2010. Flush Early and Avoid the Rush: A General Rule of Antipredator Behavior? Behavioral Ecology, 21: 440-442, 26 March. Bohorquez, Juan Camilo, Sean Gourley, Alexander R. Dixon, Michael Spagat and Neil F. Johnson. 2009. Common Ecology Quantifies Human Insurgency. Nature, 462(7275), 17 December, pp 911-914. Carlson, Jean and John Doyle. 2000. Highly Optimized Tolerance: Robustness and Design in Complex Systems, Physical Review Letters 84 (11): 2529–2532, 13 March. Carlson, Jean and John Doyle. 2002. Complexity and Robustness. PNAS 99: 2538–2545, 19 February. Csete, Marie and John Doyle. 2004. Bow Ties, Metabolism and Disease. TRENDS in Biotechnology 22(9), September. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.173.3019&rep=rep1&type=pdf http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.173.3019&rep=rep1&type=pdf Csete, Marie and John Doyle. 2010. Bow Ties, Metabolism and Disease, TRENDS in Biotechnology 22(9), September 2004. www.cds.caltech.edu/~doyle/CmplxNets/Trends.pdf. www.cds.caltech.edu/~doyle/CmplxNets/Trends.pdf Dixon, Colin, Anderson, Thomas and Krishnamurthy, Arvind, Phalanx: Withstanding Multimillion-Node Botnets, NSDI'08: Proceedings of the 5th USENIX Symposium on Networked Systems Design and Implementation, April 2008. Dove, Rick and Laura Shirey. 2010. On Discovery and Display of Agile Security Patterns. Conference on Systems Engineering Research, Stevens Institute of Technology, Hoboken, NJ, March 17-19. www.parshift.com/Files/PsiDocs/Pap100317Cser-OnDiscoveryAndDisplayOfAgileSecurityPatterns.pdf www.parshift.com/Files/PsiDocs/Pap100317Cser-OnDiscoveryAndDisplayOfAgileSecurityPatterns.pdf Dove, Rick. 2011. Patterns of Self-Organizing Agile Security for Resilient Network Situational Awareness and Sensemaking. 8th International Conference on Information Technology: New Generations (ITNG), April 11-13, Las Vegas, NV. www.parshift.com/s/110411PatternsForSORNS.pdf www.parshift.com/s/110411PatternsForSORNS.pdf Edge, Kenneth S., Gary B. Lamont, and Richard A. Raines, Multi-Objective Mobile Network Anomaly Intrusion, International Journal of Computer Science and Network Security, 6(3b):187-192, March, 2006. Forrest, S., S. Hofmeyr and A. Somayaji. 2008. The evolution of system-call monitoring. Proceedings of the 2008 Annual Computer Security Applications Conference, pp. 418-430. George, Deleep. 2008. How the Brain Might Work: A Hierarchical and Temporal Model for Learning and Recognition, PhD thesis, Stanford University. www.numenta.com/htm-overview/education/DileepThesis.pdfwww.numenta.com/htm-overview/education/DileepThesis.pdf Hambling, Dave, Drone Swarm for Maximum Harm, Defense Tech. April 10, 2006. Haack, Jereme N., Glenn A. Fink, Wendy M. Maiden, David McKinnon, and Errin W. Fulp. 2009. Mixed-Initiative Cyber Security: Putting Humans in the Right Loop. www.cs.wfu.edu/~fulp/Papers/mims09f.pdfwww.cs.wfu.edu/~fulp/Papers/mims09f.pdf Hartzog, Paul. 2010. How Different is Your Bow Tie? Blog at P2P Foundation, 21 June 2010. http://blog.p2pfoundation.net/how-different-is-your-bow-tie/2010/06/21. Hightower, R., S. Forrest and A.S. Perelson. 1996. The Baldwin effect in the immune system: Learning by somatic hypermutation. In Adaptive Individuals in Evolving Populations, R. K. Belew and M. Mitchell, (eds.), Addison-Wesley, Reading, MA, pp. 159-167. http://cs.unm.edu/~forrest/publications/baldwin.pdfhttp://cs.unm.edu/~forrest/publications/baldwin.pdf Previous Pattern References 1/2
10:38 email@example.com@stevens.edu, attributed copies permitted Hofmeyr, S. and S. Forrest. 2000. Architecture for an Artificial Immune System." Evolutionary Computation 7(1), Morgan- Kaufmann, San Francisco, CA, pp. 1289-1296. http://cs.unm.edu/~forrest/publications/hofmeyr_forrest.pdfhttp://cs.unm.edu/~forrest/publications/hofmeyr_forrest.pdf Khurana, Himanshu, Jim Basney, Mehedi Bakht, Mike Freemon, Von Welch, Randy Butler. 2009. Palantir: A Framework for Collaborative Incident Response and Investigation. In Symposium on Identity and Trust on the Internet (IDTrust), Gaithersburg, MD, April 14-16. http://netfiles.uiuc.edu/hkhurana/www/IDTrust20091.pdfhttp://netfiles.uiuc.edu/hkhurana/www/IDTrust20091.pdf Li, Aihong, et al. 2004. Utilization of Ig Heavy Chain Variable, Diversity, and Joining Gene Segments in Children with B- lineage Acute Lymphoblastic Leukemia: Implications for the Mechanisms of VDJ Recombination and for Pathogenesis. Blood, 103(12) 4602-4609, 15 June. Lockton, Dan with Davis Harrison and Neville A. Stanton. 2010. Design With Intent - 101 Patterns for Influencing Behaviour Through Design. Equifine. April. Available at http://www.danlockton.com/dwi/Download_the_cards. Lockton, Dan and David Harrison. 2009. Design for Sustainable Behaviour: Investigating Design Methods for Influencing User Behaviour. Sustainable Innovation 09: Towards a Low Carbon Innovation Revolution, 14th International Conference, Farnham Castle, UK, 26-27 October. Mahimkar, A., Dange, J., Shmatikov, V., Vin, H. and Zhang, Y., dFence: Transparent Network-Based Denial of Service Mitigation, in Proceedings of 4th USENIX Symposium on Networked Systems Design and Implementation (NSDI 2007), Cambridge, MA, April, 2007. Serre, T., Learning a Dictionary of Shape-Components in Visual Cortex: Comparison with Neurons, Humans and Machines, Ph. D Dissertation, Massachusetts Institute of Technology, June, 2006. http://cvcl.mit.edu/Papers/SerreOlivaPoggioPNAS07.pdf http://cvcl.mit.edu/Papers/SerreOlivaPoggioPNAS07.pdf Smets, Barth F. and Tamar Barkay. 2005. Horizontal gene transfer: perspectives at a crossroads of scientific disciplines. Nature Reviews Microbiology 3, 675-678 (September 2005). Troncale, L. 1978. Linkage Propositions Between Fifty Principal Systems Concepts. in Applied General Systems Research: Recent Developments and Trends : N.A.T.O. Conference Series II, Systems Science, G. J. Klir, (Ed.), Plenum Press, pp 29-52. Troncale, L. 2006. Towards A Science of Systems. Systems Research and Behavioral Science, Special Journal Edition on J.G. Miller, Founding Editor (G.A. Swanson, Ed.) 23(3): 301-321. Wilkinson, Sophie, Plants to Bugs: Buzz Off!, Chemical and Engineering News, June 30, 2001. Woese, Carl. 2000. Interpreting the universal phylogenetic tree. PNAS. 97(15):8392-6. www.ncbi.nlm.nih.gov/pmc/articles/PMC26958/pdf/pq008392.pdf www.ncbi.nlm.nih.gov/pmc/articles/PMC26958/pdf/pq008392.pdf Zhang, C., Zhang, J., Liu, S., and Liu, Y., Network Intrusion Active Defense Model Based on Artificial Immune System. Fourth International Conference on Natural Computation, Jinan, China, October 18-20, 2008. Previous Pattern References 2/2
10:39 firstname.lastname@example.org@stevens.edu, attributed copies permitted On 4-Panel Graphics The graphic depicts a classic specific example rather than a generic abstraction of key elements. This gives grounded substance to the generic concept. It may be some day that a generic abstracted graphic will be added, but it is doubtful that a specific-example graphic will be eliminated. Abstractions reduce information. The graphic is a key element of the pattern form. It captures the essence of the concept in a visual (memorable) depiction in four time-series snap-shots. The graphic depict the dynamics of situational response. Maybe: awareness, assessment, configure, respond. Maybe OODA: Observe, Orient, Decide, Act. Why four panels? 1)any more and the core essence is “likely” diffused or confused. 2)any more and it is less memorable. 3)any fewer and too much intellectual interpretation is “likely” needed. 4)any fewer and a sufficient communication is “likely” deficient. What not to do: Flow charts, SysML depictions, and other such “thinking is required” approaches.
10:40 email@example.com@stevens.edu, attributed copies permitted Core Patterns of Biological Systems (wip preliminary thinking, maybe a framework for a Pattern Language) autocatalysis (self-reproductive life itself) active infrastructure (will to live, ego, personality) horizontal meme transfer hierarchical sensemaking bow tie processor negative selection anomaly detection genetic algorithm fractal architectural reflection modules and framework This is a current conjecture, and subject to radical evolution (Early pattern work exists for the green area, nothing yet for yellow) Many more eligible: Behavior Attractors Situational Awareness Fast Learning Loops Resilience Adaptation Effortful Learning Experimental Learning 4 th Gen Warfare 5 th Gen Warfare Disposable Resources SoS Intervention/Repurposing …Etc…
10:41 firstname.lastname@example.org@stevens.edu, attributed copies permitted The respected software pioneer and computer scientist, Richard Gabriel, gives us an informative inside look at the world of software design and computer programming and the business that surrounds them. Gabriel discusses such topics as what makes a successful programming language, how the rest of the world looks at and responds to the work of computer scientists, how he first became involved in computer programming and software development, what makes a successful software business, and why his own company, Lucid, failed in 1994, ten years old. Perhaps the most interesting and enlightening is Gabriel's detailed look at what can be learned from architect Christopher Alexander, whose books-- including the seminal A Pattern Language--have had a profound influence on the computer programming community. Gabriel illuminates some of Alexander's key insights--"the quality without a name," pattern languages, habitability, piecemeal growth--and reveals how these influential architectural ideas apply equally well to the construction of a computer program. Gabriel explains the concept of habitability, for example, by comparing a program to a New England farmhouse and the surrounding structures which slowly grow and are modified according to the needs and desires of the people who live and work on the farm. "Programs live and grow, and their inhabitants-- the programmers--need to work with that program the way the farmer works with the homestead.” From the Foreword by Christopher Alexander: "What was fascinating to me, indeed quite astonishing, was that in Gabriel's essays I found out that a computer scientist, not known to me, and whom I had never met, seemed to understand more about what I had done and was trying to do in my own field than my own colleagues who are architects." Free full-book download from the author’s website: http://dreamsongs.com/Files/PatternsOfSoftware.pdf
10:42 email@example.com@stevens.edu, attributed copies permitted Eventual Aspiration for our Language of Patterns From Richard P. Gabriel. 1996. The Quality Without A Name (essay), in Patterns of Software, Oxford Univ. Press, pp. 33-43. http://dreamsongs.com/Files/PatternsOfSoftware.pdf Alexander: “I was no longer willing to start looking at any pattern unless it presented itself to me as having the capacity to connect up with some part of this quality [the quality without a name]. Unless a particular pattern actually was capable of generating the kind of life and spirit that we are now discussing, and that it had this quality itself, my tendency was to dismiss it, even though we explored many, many patterns. “It is a subtle kind of freedom from inner contradictions.” “…it became clear that the free functioning of the system did not purely depend on meeting a set of requirements. It had to do, rather, with the system coming to terms with itself and being in balance with the forces that were generated internal to the system, not in accordance with some arbitrary set of requirements we stated.” A system has this quality when it is at peace with itself, when it has no internal contradictions, when it is not divided against itself, when it is true to its own inner forces. And these forces are separate from the requirements of the system as a whole. Alexander proposes some words to describe the quality without a name, but even though he feels they point the reader in a direction that helps comprehension, these words ultimately confuse. The words are alive, whole, comfortable, free, exact, egoless, and eternal. I’ll go through all of them to try to explain the quality without a name. [Read (Gabriel 1996: 33-43) for details, or (Alexander. 1979. The Timeless Way of Building. Oxford Univ. Press]
10:43 firstname.lastname@example.org@stevens.edu, attributed copies permitted What do sexually transmitted diseases, the World Wide Web, the electric power grid, Al Queda terrorists, and a cocktail party have in common? They are all networks. They conform to surprising mathematical laws which are only now becoming clear. Albert-Laszlo Barabasi has helped discover some of those laws over just the past five years, and though they are some pretty abstruse mathematics, he has written a clear and interesting guide to them. Not only has he attempted in this book to bring the math to non- mathematicians, he has shown why the work is important in down-to-earth applications. It is important for those multitudes who have no taste for math to know that this is not a book full of equations; Barabasi knows that for most of his readers, doing the math is not as important as getting a feel for what the math does. He explains the basic history of network theory, and then shows how his own work has turned it into a closer model of reality, a model that most of us will recognize. Networks are all around us, and they are simply not random. Some of our friends, for instance, are loners, while others seem to know everyone in town. Some websites, like Google and Amazon, we just cannot avoid clicking on or being referred to, but many others are obscure and you could only find them if someone sent you their addresses. Barabasi calls these "nodes" with such an extraordinary number of links "hubs," and he and his students have found laws of networks with hubs, showing such things as how they can continue to function if random nodes are eliminated but they fragment if the hubs are hit. Barabasi is currently doing research to show what intracellular proteins interact with other proteins, and true to form, some of them are hubs of reactions with lots of others. Finding the hubs of cancerous cells, for instance, and developing ways of taking them out, show enormous promise in the fight against cancer. And finding the hub terrorists in Al Queda in order to take them out would be the best way to eliminate the network. [Amazon reviewer Rob Hardy]
10:44 email@example.com@stevens.edu, attributed copies permitted There is a lot of good stuff in here. The descriptions of the patch procedure and simulated annealing, for instance, are very nice. This book can be useful to the motivated general reader, and to a scientist who wants to see the very basics of some novel ideas. It can also be useful for those familiar with complexity as an account of how different pieces fit together. It's important to remember that the book is not a text in, say, biochemistry. Rather, it's about a way to see the world. At this stage of the idea development life cycle and in a basic treatment like this, it would be counterproductive to insist that these modeling tools reproduce everything we know or start at the level of complication of a mature science. If the book deals in toy examples that relate to a different view for pieces of the world and how they relate, it has done most of its job. [Amazon reviewer]
10:45 firstname.lastname@example.org@stevens.edu, attributed copies permitted Review By C. W. Richards (Atlanta, GA United States) C. W. Richards Global guerrillas practice something Robb calls "open source warfare," which means that in the modern environment, people even on different continents can form or join groups, train, and carry out operations much more quickly than in the past or than the major legacy states can today. As the groups learn from each other (and a sort of Darwinism selects out the unfit), a larger pattern forms, an "emergent intelligence," similar to a marauding colony of army ants, no one of which is very sophisticated, but operating together according to simple rules, they are survivable, adaptable, and in a suitable environment, invincible. To construct this model, Robb employs a number of concepts that may be new to people unfamiliar with modern systems theory: close-coupled systems, self-organization, emergent properties (particularly "intelligence"), stigmergy, and the concept of complexity arising from simple processes. He also introduces new tools for understanding how systems work in the modern world: open source insurgency, global virtual states, superempowerment, systempunkts, and "black swans." Robb's general strategy is to improve resilience by any means possible. I could imagine, for example, that instead of building new power plants that, along with their distribution systems, are vulnerable to disruption, the government provides market incentives to improve resilience. The government could increase subsidies to utilities and require all of them to buy electricity from homeowners during the day and sell it at reduced rates at night. As more people add power generation capability to their houses - solar, wind, geothermal, hydroelectric, whatever - resilience improves. This may not be the most efficient solution, but in the age of open source insurgency, too much efficiency can be dangerous. Robb makes a compelling case that this model will also work for national security. It is certainly working very well for the groups we are fighting. Order emerges from chaos - ready or not Publisher: Wiley (April 20, 2007)
10:46 email@example.com@stevens.edu, attributed copies permitted Through a very insightful book, Peter Miller turns to Nature to explain crowd behavior. Leveraging upon numerous scientific studies, Peter elaborates the principles through which even insects with low individual intelligence perform extraordinary feats of brilliance as a group. That too without hierarchy or elaborate rules! Peter Miller calls this intelligent group behavior - the smart swarm. He then explains how the smart swarm works - using biology to unlock the secrets of collective behavior. The dangers of group behavior are also brought out through the examples of locusts - which is useful to understand how human groups also sometimes turn violent. What are the principles of smart swarms? The first principle of a smart swarm is self organization. Through the basic mechanisms of decentralized control, distributed problem solving and multiple interactions, members of a group without being told can transform simple rules of thumb into meaningful patterns of collective behavior. This is explained through the functioning of ant colonies - that is "Though Ant's aren't smart, why Ant colonies are?“ The second principle of a smart swarm is 'diversity of knowledge' - which is basically achieved through a broad sampling of the swarm's options, followed by a friendly competition of ideas. Then using an effective mechanism to narrow down the choices, swarms can achieve 'wisdom of crowds'. The honeybees example of choosing a new nest illustrates this very clearly - and Peter shows how communities and businesses can build trust and make better decisions by adapting this. The third principle is indirect collaboration. If individuals in a group are prompted to make small changes to a shared structure that inspires others to improve it even further, the structure becomes an active player in the creative process. This is explained beautifully with the example of how termites build huge structures. We also see this in our internet world through Wikis!!! The fourth principle is adaptive mimicking. With the example of flight behavior of starlings, Peter shows how the basic mechanisms of coordination, communication and copying can unleash powerful waves of energy or awareness that race across a population evoking a feeling of mental telepathy. [Amazon reviewer Sam Santhosh ] Publisher: Avery (August 5, 2010)
10:47 firstname.lastname@example.org@stevens.edu, attributed copies permitted “Well known in statistical circles, Bayes’s Theorem was first given in a posthumous paper by the English clergyman Thomas Bayes in the mid- eighteenth century. McGrayne provides a fascinating account of the modern use of this result in matters as diverse as cryptography, assurance, the investigation of the connection between smoking and cancer, RAND, the identification of the author of certain papers in The Federalist, election forecasting and the search for a missing H-bomb. The general reader will enjoy her easy style and the way in which she has successfully illustrated the use of a result of prime importance in scientific work.”— Andrew I. Dale, author of A History of Inverse Probability From Thomas Bayes to Karl Pearson and Most Honorable Remembrance: The Life and Work of Thomas Bayes Copyright 2011 Not an official text – but highly recommended
10:48 email@example.com@stevens.edu, attributed copies permitted Systems Analysis: The Brain A Pattern Memory and Prediction System The brain constantly compares new sensory information with stored memories and experiences and combines the information to anticipate the future. In essence, as we wander around, we build a reserve of information from which we construct an internal model of the world. But we constantly update that model. The continuous interplay of sensory input, memory, prediction and feedback—which occurs instantly through parallel processing in the neocortex—ultimately gives rise to consciousness and intelligence. Hawkins proffers a "comprehensive theory of how the brain works," of "what intelligence is," and of "how your brain creates it." This book provides some provocative thoughts on how the brain and the mind may actually function. Richard Lipkin, Scientific American Hawkins is a founder of two leading mobile computing companies—Palm Computing and Handspring—and also of the Redwood Neuroscience Institute, which explores memory and cognition, and now…Numenta, which is developing artificial cortex. Not an official text – but highly recommended
10:49 firstname.lastname@example.org@stevens.edu, attributed copies permitted Photo: Ethan Hill Scientific American, Aug 2006 The Expert Mind Studies of the mental processes of chess grandmasters have revealed clues to how people become experts in other fields as well. Effortful study is the key to achieving success in chess, classical music, soccer and any field of expertise. Research indicates that motivation is a more important factor than innate ability. 200,000 patterns, 10,000 hours
10:50 email@example.com@stevens.edu, attributed copies permitted Infrastructure evolution Assembly in SIL Module mix evolution Module readiness Infrastructure Modules Rules/Standards Integrity Management Active Passive process engineer production system engineer material manager small upgrade tech refresh large re-fit QRC Device/Power/Cooling Installation Architecture boxes rackszones SILs aircraft hardware Sockets Signals Security Safety Service Physical interconnect standards Data/power/cooling transmission Personnel/Sil/supply-chain/et al. Weight/space/installation rules Agile system/process ConOps Boss, Jason and Rick Dove. 2010. Agile Aircraft Installation Architecture In a Quick Reaction Capability Environment. INCOSE International Symposium, Chicago, July 12-15. www.parshift.com/Files/PsiDocs/Pap100712IS10-AgileAircraftInstallationArchitecture.pdfwww.parshift.com/Files/PsiDocs/Pap100712IS10-AgileAircraftInstallationArchitecture.pdf
10:51 firstname.lastname@example.org@stevens.edu, attributed copies permitted Task Backlog Buffer Infrastructure evolution: Task assembly: Task elements: Task readiness: Infrastructure Standards Integrity Management Active Passive Last Planner Process Manager Supes/Foreman Project Manager Supes/Foremen/Expediters Last Planner Agile Project Management www.parshift.com/s/130624Last Planner.pdf www.parshift.com/s/130624Last Planner.pdf Active management of the anticipated schedule and work flow to ensure there is always a buffer of “quality” jobs ready to work on and matched with resources. master sched CPM tasks materials tools Agile architecture Pattern based on: (Ballard 1997) Lookahead Planning: the Missing Link in Production Control (Ballard 1998) Shielding Production: an Essential Step in Production Control (Ballard 1999) Improving Work Flow Reliability (Ballard 2000) The Last Planner System of Production Control-PhD Thesis equipment production units activity definitions Key Practices: Rules 1-2-3 and Lookahead Make ready Learn & Correct Work Task Task Lookahead Window weekweekweekweekweekweek 654321 Components Task Soundness/Sequence/Size Task Definitions Physical Site Security Construction Safety Standards/Regs Master Sched, Learning, R1-2-3 Sockets Signals Security Safety Service MS Learning Change
10:52 email@example.com@stevens.edu, attributed copies permitted BSAs DepartmentsSSAsContractors COTS Apps ETLs & BIMs Infrastructure evolution System assembly/reconfiguration Module mix evolution Module readiness Infrastructure Phase 2: Desired Phase 3: Refined Components/Modules Rules/Standards Integrity Management Active Passive Prog Mgr Dept User Proj Mgr BSAs Silterra Agile ERP – Development System Sockets Signals Security Safety Service Scrum-Like Team Collaboration Scrum-Like Progress/Needs Supply Chain Protection (Team) No Req Changes Development ConOps/Rules Phase 1: Out of Box
10:53 firstname.lastname@example.org@stevens.edu, attributed copies permitted COTS ERP Apps Custom Other Apps COTS Other Apps App ETLs Data Bases Custom ERP Apps Infrastructure evolution System assembly/reconfiguration Module mix evolution Module readiness Infrastructure Customer MyFab Components/Modules Integrity Management Active Passive SSAs Dept Users & BSAs BSAs System examples are SOA-like instances of departmental needs Silterra Agile ERP – Developed System Planning/SchedulingEOM Financial Rpt Rules/Standards ETL Template Sockets Signals Security Safety Service API, ETL, BIM, ESB Initial XML Protocol SEA -Appropriate Strategy Pub/Sub Bus Architecture/ConOps Rules
10:54 email@example.com@stevens.edu, attributed copies permitted HH PNM Agile Substation System Design Development www.parshift.com/Files/PsiDocs/Pap080404Cser2008DevOpsMigration.pdf engineers switchgeartransformers termination structures low-voltage feeders station steel Infrastructure evolution System assembly Component evolution Component readiness Infrastructure H Station Fly-Thru StationT Station Components Rules/Standards Integrity Management Active Passive chief engineer design engineer DASL program mgr min/max purchaser T T H H H TT Agile Architectural Pattern Diagram Sockets Signals Safety Security Service DASL module interconnects Power flow Construction policies/regs No development customization DASL design tool ConOps H-pad standards Fly-pad standards
10:55 firstname.lastname@example.org@stevens.edu, attributed copies permitted Concepts That Enable Agility to have awareness to take action Agility System Response Architecture Knowledge Management Culture of Change Proficiency Reusable principles of Reconfigurable Peer-Perr Interaction Deferred Commitment Self Organization Distributed Cont & Info principles of Scalable Modularity Facilitated Reuse Plug Compatibility Evolving Framework Redundancy & Diversity Elastic Capacity with domains of ProActive Change Creation Improvement Migration Modification ReActive Change Correction Variation Expansion Reconfiguration with domains of Change Metrics Time Cost Quality Scope with molded by language of Value Propositioning consists of practices and processes for Response Ability Change Management enabling change with to select actions
10:56 email@example.com@stevens.edu, attributed copies permitted Early Conceptual Work Enables and Constrains System Possibilities "…the development of a basic idea and the first embodiment of the idea; these two initial activities are often called invention and are usually not part of the engineering of a system…" Dennis Buede We will focus on this area Buede's book addresses the procedures and processes that turn concept into reality. That is a different course.
10:57 firstname.lastname@example.org@stevens.edu, attributed copies permitted Setting Expectations All slides in course material will not be reviewed/presented/discussed, …they are there to draw upon as appropriate, and for later reference. Some slides not in the regular course material will be employed, …for case studies as emerging interests indicate. Some slides are very dense and not screen-viewable at a distance, …they augment the text with reference material viewable on your PC. This is not a Systems Engineering Process course, …those are available under appropriately different course titles. This course focuses on design and engineering concepts, …that enable responsive/adaptable systems. Various thinking-tools and thinking-disciplines are introduced, …thinking fruitfully is a creative activity, not a procedure. This is not a software-systems engineering course, …nor focused on any other specific systems domain.
10:58 email@example.com@stevens.edu, attributed copies permitted IT TAKES TWO TO TANGO FEEDBACK TIME
10:59 firstname.lastname@example.org@stevens.edu, attributed copies permitted Grading (For-Credit Students) 10% on class participation: Peer review presentations: demonstration of relevant knowledge application. Peer review contributions: collaborative engagement with projects of others. Evidence of study: knowledgeable reference to the readings. 30% on operational model – Midterm deliverable Two-page operational story: clear evidence of an agile system in operation demonstrated with response objectives, requirements, values, response enabling principles, and operational/integrity management. Three-element response ability model: relevance and clarity of key concepts in RS Analysis, RRS Principles, and Architectural Concept Pattern diagram. Evidence of study: knowledgeable reference to the literature and readings. 60% on conceptual design report – Final deliverable Articulate a comprehensive new conceptual design, or analysis of an existing design: response objectives, issues with metrics, and enabling principles; strategic themes and activity web; closure matrix with descriptions; and operational management and responsibilities – see 678 Project Guidance document for the definitive word. Evidence of study: knowledgeable reference to the literature and readings. Reality:The first deliverable is key. Your true understanding of necessary fundamentals is illuminated here. Feedback on this will put your train back on the rails. due nlt Monday 2 weeks after class due nlt Monday 6 weeks after class
10:60 email@example.com@stevens.edu, attributed copies permitted Minimum: 80 Hrs Outside-of-Class Work 10-20 Hrs reading the text book 30-20 Hrs researching and noodling 40 Hrs composing and writing You are Graduate Students A – Thoughtfully engaged with demonstrated application-design understanding B – Read, followed instructions, applied tools, demonstrated utility understanding C – Any of: blew it off, no understanding of basic concepts demonstrated, didn’t complete the closure matrix and discussion or other basic project steps. ---- This is about: how your system addresses surprises (primary) not about what your system does functionally (secondary) Key: When it clicks…that drag-and-drop, plug-and-play (operational activity) is enabled by “encapsulated” module and “evolving” framework, and that you have this all around you in your life…and you already know it well: Providing dinner for unexpected guests Assembling a team for a task Appreciating your football team in action Reconfiguring your home entertainment system or your PC Strawman budget
10:61 firstname.lastname@example.org@stevens.edu, attributed copies permitted "Make no little plans; they have no magic to stir men's blood and probably will themselves not be realized. Make big plans; aim high in hope and work, remembering that a noble, logical diagram once recorded will not die, but long after we are gone will be a living thing, asserting itself with ever-growing insistency" [Daniel Burnham, architect]. Explore the Possible for Your Term Project
10:62 email@example.com@stevens.edu, attributed copies permitted Course Project (For-Credit Students) (always refer to www.parshift.com/AgileSysAndEnt/ProjGuide/678ProjGuideCurrent.pdf for current requirements)www.parshift.com/AgileSysAndEnt/ProjGuide/678ProjGuideCurrent.pdf A newly built custom assembly line for each and every small-batch run, every time, just in time. Life with System X – Agility in Action By Rick Dove, Paradigm Shift International, e-mail: firstname.lastname@example.org, 505-586-1536, Senior Fellow, Agility Forum Look through Fred Mauck's eyes for a moment. You work in a GM stamping plant outside of Pittsburgh that specializes in after-model-year body parts. Your principal customer is GM's Service Parts Organization. They might order '73 Chevelle hoods quantity 50, '84 Chevy Impala right fenders quantity 100, or '89 Cutlass Supreme right front doors quantity 300. Your plant stamps the sheet metal and then assembles a deliverable product. Small lots, high variety, hard-to-make-a-buck stuff. Every new part that the plant takes on came from a production process at an OEM plant that occupied some thousands of square feet on the average; and the part was made with specialized equipment optimized for high volume runs and custom built for that part geometry. To stamp a new deck lid (trunk door) part you bring in a new die set - maybe six or seven dies, each the size of a full grown automobile, but weighing considerably more. And you bring in assembly equipment from an OEM line that might consist of a hemmer to fold edges of the metal, perhaps a pre-hemmer for a two-stage process, dedicated welding apparatus for joining the inner lid to the outer lid, adhesive equipment for applying mastic at part-specific locations, piercer units for part-specific holes, and automated custom material handling equipment for moving work between process workstations. You got a call a few weeks ago that said your plant will start making the Celebrity deck lids, and production has to start in 21 days. Not too bad - sometimes you only have four days. For new business like this your job is to get the necessary assembly equipment from the OEM plant, reconfigure the equipment and process to fit your plant, and have people ready to produce quality parts in the next three weeks. Others are responsible for the die sets and stamping end of the production process. In the last 12 months this happened 300 times. In the last five years you've recycled some 800,000 square feet of floor space in OEM plants for new model production. At this point you have assembly equipment and process for some 1000 different parts - but no extra floor space ever came with any of it. high-variety production - in a business that is traditionally based on high volume economics - and you've learned to do it without the usual capital budget. Eight years at this has evolved some pretty unique techniques - and a pretty unique culture as well. You don't do this by yourself - you're a team leader that may use almost anyone from anywhere in the plant. At this point almost everyone is qualified to help bring in new work - surviving under these conditions has developed a can-do/let- me-at-it attitude almost everywhere, and a shared understanding of how to do it. Eight years ago the plant went to a single job classification in production, cross training everyone on everything - a press operator one day might change dies as well, the next day work in the assembly area building hoods in the morning and fenders in the afternoon - and the following day go off to another plant to review a piece of equipment or part for how to bring it back. For this new business Jim Lesniewski wanted to do the initial recon. He went on the last trip too, experimenting with his video camera. Now he thinks he's ready to do a perfect taping job. He got the idea himself while trying to bring several jobs at once back from another GM facility. This environment encourages self initiative. In addition to taping the operational assembly process he added close-ups of key equipment pieces this time. In the debrief review everyone saw the same thing at the same time - there was almost no debate over what to bring back and what to ignore - and you got a jump on the equipment modifications by seeing what was needed in advance. Some time ago the value of having a good cross section represented in these reviews became evident: nobody gets surprised, everyone shares their knowledge, and when the eqchine, two welding robots, the welding fixtures, two press piercers, the shuttles, the press welders, and the three automated material handling fixtures. Basically bringing back a foot print of 200 square feet from a process that covered 2500 square feet. The rest will go to salvage disposition while the hemmer goes to "hemmer heaven" - that place in your plant where some 200 different hemmers hang out until needed. That you only need the hemmer is where a key part of the plant's unique core competency comes to play. Rather than build a growing variety of product on some Problem/Opportunity Response Objectives Response Issues/Metrics Strategic Activity Web Architecture & Integrity Applied Principles Closure Matrix Conclusion & References Detailed Conceptual Design Documentation ---------------- Comprehensive to one Skilled in the Arts Response Ability Model 3 MS PowerPoint Slides 5 Page Operational Model - Due as deliverable #1 Operational Story ~ 2 MS Word Pages ~ 20-30 Pages Due as Deliverable #2 Includes strategic objectives/themes Operational Story RSA - JIT Assembly Lines RRS - JIT Assembly Lines ACP - JIT Assembly Lines AAP
10:63 email@example.com@stevens.edu, attributed copies permitted Creating Conceptual Design Closure The closure tool is where design thought gets deep. Here the preliminary issues, principles, and activities are sifted for relevance and related for synergy. The tool is first used to specify which activities will address which issues, and why; and to verify (in the mind of the designer) that the set of issues and the set of activities are necessary and sufficient. It is a time to step back from the preliminary, somewhat brainstormed, formulation of the problem and the solution- architecture, and do a sanity check before specifying design-principle employment. Not explored further here, Chapter 7 of the text book can assist. The real work with the closure tool is generally on the employment and purpose of principles - the ones that would compromise potential if they are not employed as design elements. Issue-Focused, Principle-Based Design – Discussion 1)Pick an activity, and describe its general process sequence steps. 2)Do a paragraph for each issue that the chosen activity addresses, and show in that paragraph how the principles are employed to address the issue. 3)The chosen activity must have enough issues and principles to demonstrate your understanding, else pick a second activity and do that as well. 4)See chapter 7 in the text book for an example of what is expected.
10:64 firstname.lastname@example.org@stevens.edu, attributed copies permitted Finding Reference Papers Stumble around: Google keywords, try different variations, try general terms with the same meanings. Getting relevance: after finding one or a few that are relevant… - Look at the papers they reference for some older papers that may be relevant. - Google the name of the papers you think are relevant and find some newer papers that reference them. - Often papers that are pay-for-view can be found for free on the author’s personal site or on some other site – Google the paper name and use the advanced option to find PDF files only. News stories, blogs, and other web-only references may have some value, but cannot be your principle sources of support – they are generally inaccurate and often untrue. Use a proper, complete, and consistent form for all references and citations, such as Chicago Style. Anything less is equivalent to demonstrating verbal illiteracy as an orator.
10:65 email@example.com@stevens.edu, attributed copies permitted Chicago Style Citations www.lib.berkeley.edu/instruct/guides/chicago-turabianstyle.pdf Chicago style allows you to choose between two systems of providing references: 1. Notes and bibliography: numbered footnotes or endnotes in your text, with Bibliography or Works Cited list at the end of the paper, listing alphabetically the sources in your notes. 2. In-text author-date citations and reference list: in your text, brief parenthetical references consisting of the author's last name, publication year, and page(s) referred to, with an alphabetized Reference List at the end of your paper providing complete entries for works cited in parenthetical references. ---------- Use Chicago Style for Term Paper: http://wwwlib.murdoch.edu.au/find/citation/chicago.htmls http://wwwlib.murdoch.edu.au/find/citation/chicago.htmls http://www.chicagomanualofstyle.org/tools_citationguide.html
10:66 firstname.lastname@example.org@stevens.edu, attributed copies permitted Getting it Right Download Unit 11 and read it – it is a small collection of the slides from the 10 class units that pertain to the expectations for mid-term and final deliverables. Read the Project Guidance document again before you start your mid-term and final project, and again before you submit your mid-term and final project Unit 11
10:67 email@example.com@stevens.edu, attributed copies permitted Avoid This Final-Project Feedback 1)Everything noted in the 4-week deliverable things to avoid applies to 10-week deliverable. 2)Comments made in the 4-week deliverable feedback were not addressed. 3)Activities sound like things (nouns) rather than processes (verbs). 4)Closure Matrix Discussion: You didn’t do it like it was asked for: pick one single activity, detail it’s process steps, then do a paragraph for each issue that the chosen activity addresses and show in that paragraph how the principles are employed to address the issue. The chosen activity must have enough issues and principles to demonstrate your understanding, else pick a second activity as well. See chapter 7 in the text book. 5)Closure Matrix: Your issues are very specific features/solutions rather than generally stated response problems that the “activities” address with the employment of ‘principles’. There is a hierarchy in the closure matrix concept: On one extreme, issues are totally general. On the other extreme, the employment of principles is very specific. Activities are the intermediate “mechanism” for applying engineering principles in the specific solution of a general problem. IF THIS IS CONFUSING…NOW IS THE TIME TO CLEAR IT UP
10:68 firstname.lastname@example.org@stevens.edu, attributed copies permitted Avoid This Final-Project Feedback 6) Your final deliverable does not include one or more of the following: - Operational story - RA Analysis of proactive and reactive operational-response issues - RRS Principles in bulleted application - Architectural Concept Pattern diagram - ConOps Strategic Objectives and Activities web - Reality Factors - Integrity Management Discussion of four responsibilities - Closure Matrix relating activities, issues, and principles – with discussion. - Conclusion 7)You are confused about the differences between, and proper use and form of, references, footnotes/endnotes, and bibliographies. 8)This is supposed to be graduate-level work. You need to have someone proof your work for intelligible use of English and/or excessive typos. 9) All tables with bulleted items are to be explained in the text so the concept referred to by the bullets are understandable by the reader. 10)You ignored proper and consistent reference and/or citation format. IF THIS IS CONFUSING…NOW IS THE TIME TO CLEAR IT UP
10:69 email@example.com@stevens.edu, attributed copies permitted On Writing Papers Understand how to convey the value and purpose of your work. Understand correct use and form of references and citations. Understand the need and purpose of a literature review. Understand the purpose and nature of an abstract. Understand the various kinds of plagiarism. Understand your reviewer’s criteria. Read peer-reviewed published papers and note how they deal with these issues. Term papers are expected to be near conference quality. Master’s paper’s are expected to be peer-reviewed conference/journal-quality. For a good tutorial (40-60 minutes) on academic and Journal-quality papers, see INCOSE Webinar #7 by Brian Sauser at www.incose.org/practice/webinars.aspxwww.incose.org/practice/webinars.aspx
10:70 firstname.lastname@example.org@stevens.edu, attributed copies permitted The Professor’s Model Objective: 1)Cause insightful understandings of permanence 2)Instigate an open community of employment and extension Belief: 1)The concepts are natural and all around us, and are already viscerally understood 2)Many types of barriers can inhibit explicit understanding Short goal:Rock solid understanding of drag-n-drop, plug-n-play as architecture of encapsulated modules and evolving infrastructure Long goal:Appreciation and utility of the other 8 principles develops naturally Strategy:1)Exposure to a wide variety of examples 2)Fast drill-and-practice exercises with critical feedback 3) Discover and overcome individual assimilation barriers Assumption:The student is equally engaged Commitment:I will help anyone who believes there is a pony in here, … and shows equal commitment
10:71 email@example.com@stevens.edu, attributed copies permitted Agility Must Address Today’s Needs First; Bringing Agile Benefits to Current Operating Modes Now, While Enabling New Operating Modes for Tomorrow. “It must be considered that there is nothing more difficult to carry out nor more doubtful of success nor more dangerous to handle than to initiate a new order of things; for the reformer has enemies in all those who profit by the old order, and only lukewarm defenders in all those who would profit by the new order; this lukewarmness arising partly from the incredulity of mankind who does not truly believe in anything new until they actually have experience of it.” Nicolo Machiavelli, The Prince
10:72 firstname.lastname@example.org@stevens.edu, attributed copies permitted EXERCISE 1) Read the Project Guideline Document 2) Start an MS Word document called: 678D0- V1.doc (D0 signifies deliverable zero) 3) In this document: a)Declare your project intention and title (same as or different than D1) b)Outline section headings of your project report per guidelines you read c)embed your system’s Strategic Activity Web bubble chart, or from it list your objectives and activities (precise wording is critical) d)Transfer this document to the instructor now (email or memory stick)
10:73 email@example.com@stevens.edu, attributed copies permitted Master’s Project Guidelines http://sse.stevens.edu/fileadmin/sse/academics/resources/Masters_Project_Guidelines.pdf Project Intent and Objectives: The intent of the Master’s project is for the student to apply the systems engineering concepts, principles and practices they have learned to: 1.A “real” problem within an application domain of interest to them, their sponsor and their assigned academic advisor; 2.An investigative study of some aspect of systems engineering (e.g., the utility and relevance of an ontology to project success); or 3.The development of a systems engineering application case study. The objective of the project is to “stretch” the thinking of the student and to provide an experience in applying theoretical concepts within a pragmatic domain. Although the student may apply only a subset of their coursework in the execution of the project, he or she must conduct some form of independent research to be successful in this endeavor. The specific approach to completing the requirements of the project will depend on the nature of the project (application, SE research, or case study) selected by the candidate and the faculty advisor. In the event of a “real” application within a domain of interest, some sponsors have offered the mentorship of subject matter or domain experts to complement faculty support and advising. Independent of the project focus and emphasis, the output is to be in the form of a technical paper suitable for submission to a referred conference or technical journal. It is expected that students will spend approximately 150-200 hours in the completion of the project.
10:74 firstname.lastname@example.org@stevens.edu, attributed copies permitted Now…before we part… private or open consultation on project thoughts, as you wish