Presentation is loading. Please wait.

Presentation is loading. Please wait.

UW EE Colloquium Feb. 20091 Network Simulation and Emulation in a Testbed Era Tom Henderson University of Washington The Boeing Company UW EE Colloquium.

Similar presentations


Presentation on theme: "UW EE Colloquium Feb. 20091 Network Simulation and Emulation in a Testbed Era Tom Henderson University of Washington The Boeing Company UW EE Colloquium."— Presentation transcript:

1 UW EE Colloquium Feb Network Simulation and Emulation in a Testbed Era Tom Henderson University of Washington The Boeing Company UW EE Colloquium February 24, 2009

2 Engineering, Operations & Technology | Boeing Research & TechnologyNetworked Systems Technology Copyright © 2009 Boeing. All rights reserved. Information Assurance Digital Communications & Network Technology Information Management and Transformation Applied Mathematics Architecture and Networked Systems Technology Contract Research and Development Programs Boeing Research & Technology organization Development of high assurance multi-level security technologies –EAL7 common criteria evaluation experience –Formal methods in support of EAL 7 evaluation High assurance guard solutions for edge, core, & tactical environments Security modeling & formal method analysis of end to end systems Policy-based secure computing –Systems of Systems IA approach Intrusion detection correlation technologies Secure workflow and SOA technologies Information operations System security engineering Advanced trusted system implementations Text analysis for information assurance Proprietary program support in IA technologies Secure RFID infrastructure Anti-tampering technologies Networked Systems Technology Phased array antennas Free air laser communications High capacity RF communications Bandwidth efficient modulation On board wireless communications Network processors AFRL RAPID Integrated communications systems MANET protocols Reliable routing Network management for tactical environments Auto-configuration Network gateways End to end QoS Network security UAV network router Data & Text Mining Human Language technologies Decision support information Information visualization Machine translation Knowledge representation Data integration Simulation visualization 3D visualization Model-based collaboration Social networks analysis Intelligent graphics & docs. Collab. human robot systems Autonomous systems –Architectures –World modeling –Multi-vehicle behaviors Reasoning Adaptive planning Learning Distributed multi-agents Global information sharing RFID info. mgmt. services Adaptive intelligent info. integration Semantic modeling, ontologies, adaptive workflow modeling Quality of Service info. algorithms Wireless mobile dynamic info. mgmt. Sensor networks Data & information fusion Info. assessment aids Trends analysis/intent inferencing Situation assessment Resource allocation techniques Deconfliction algorithms Course of action analysis/generation Variation analysis & reduction Reliability & time dependent data analysis Analysis of large datasets Financial modeling & decisions Survey & test design Statistical training Quality assurance & Lean+ Design of Experiments Computer-aided geometric design Numerical optimization Design space exploration Multidisciplinary Design Optimization Geometric modeling Multivariate data fitting Optimal control Engineering modeling & analysis Computational mechanics Sensor Fabric design & operation System dynamics & control modeling Data structures for complex product families Kalman filter applications Operations research Systems of Systems modeling & analysis Discrete/network/stochastic/ Systems of Systems optimization Data fusion Discrete-event simulation Transportation, manufacturing, business, communication & military system modeling UML architecting Arch. analysis CONOPS development System requirements NR-KPP compliance SoSE Systems engineering NCO demo. LVC technologies LVC tools DIS router Data link translation Gateways Networked manufacturing Location services Planning / visibility tools Functional integration High performance computing GRID computing Resource management Functional integration CINS lab System/network mgmt. QoS Mobile agents Situation awareness Reconfigurable computing S/W architecture S/W processes Arch. evaluation (ATAM) Performance assessment (PAPM) QoS Real-time system development Tactical info. mgmt. with an emphasis in C2/tactical collaboration Quality of Service network utilization Software enabled control for UAV automation Dynamic network management Affordable validation & verification of flight critical software & systems Collaborative anticipatory systems Cyberwarfare / Cybercraft Effects-based operations System & vehicle health management Diagnostics InfoCenter & information management Database technology & implementation Distributed sensor system technology Home organization

3 UW EE Colloquium Feb What is a network simulator? A software tool and set of models used to evaluate computer network behavior A discrete-event network simulator describes networks in terms of a sequence of discrete events –The simulation kernel temporally orders and dispatches events to models –Models generate output or more events –Time advances in discrete steps

4 UW EE Colloquium Feb Application The basic model Application Protocol stack Node NetDevice Application Protocol stack Node NetDevice Channel Packet Simul

5 UW EE Colloquium Feb The basic model // Define your topology Node n0 = new Node; Node n1 = new Node; AddInternetStack (n0, n1); Channel c0 = new Channel;. Connect (n0, n1, c0); Application a0 = new TrafficGenerator; // Configure things a0.SetDataRate (1Mb/s); a0.Start (10.0 seconds); // Define outputs WriteTraceFile (“outfile”); // Run the simulator Simulator::Run();

6 UW EE Colloquium Feb Network simulation remains popular Many commercial products –e.g. OPNET, QualNet, Shunra VE, Matlab Simulink, OMNet++, NCTUns, SSFNet, Extend A few free, open source projects –ns-2 is the most widely used such tool –roughly 10,000 downloads/month Widely used in research Source: Search of ACM Digital Library papers citing simulation,

7 UW EE Colloquium Feb Skepticism in the research community “For years, the community had to rely on simulators, which now seem a little dated, and it’s not clear who was convinced to adopt anything new based on ns2 simulations;” Nick McKeown, VINI public review, ACM Sigcomm 2006

8 UW EE Colloquium Feb What has changed? Simulators have historically been used for experiments in place of real machines –A tradeoff of realism for scale But now... Cost of test hardware has plummeted Virtualization technologies proliferate

9 UW EE Colloquium Feb Emulab (University of Utah) Source:

10 UW EE Colloquium Feb Emulab Original Emulab allowed experiments of roughly ~100 nodes –FreeBSD Dummynet and VLANs to interconnect the nodes Emulab now offers virtualization –Experiments of up to ~2000 (virtual) nodes –Requires new approaches for experimental configuration Also, some wireless nodes Source: M. Hibler et al., “Large-scale Virtualization in the Emulab Network Testbed,” Proc Usenix Technical Conference.

11 UW EE Colloquium Feb PlanetLab: Global-scale Internet testbed Source: PlanetLab “slices”

12 UW EE Colloquium Feb PL-VINI and Trellis PL-VINI adds virtual networking capabilities (via network overlay) to PlanetLab –A. Bavier et al, “In VINI Veritas: Realistic and Controlled Network Experimentation,” Proc. ACM Sigcomm, 2006 Trellis is a kernel-based implementation of the same concept –S. Bhatia et al, “Trellis: A Platform for Building Flexible, Fast Virtual Networks on Commodity Hardware,” ROADS Workshop, Dec. 2008

13 UW EE Colloquium Feb ORBIT (Rutgers WINLAB) Source: a/b/g radios Bluetooth GNU/USRP radios Noise generators

14 UW EE Colloquium Feb Simulation “out of the loop” Source: NSF GENI project:

15 UW EE Colloquium Feb Skepticism in the research community “For years, the community had to rely on simulators, which now seem a little dated, and it’s not clear who was convinced to adopt anything new based on ns2 simulations;” Nick McKeown, VINI public review, ACM Sigcomm 2006

16 UW EE Colloquium Feb Overheard* on e2e-interest mailing list “...Tragedy of the Commons...” “...around 50% of the papers appeared to be... bogus...” “Who has ever validated NS2 code?” “To be honest, I'm still not sure whether I will use a simulation in a paper.” “...I will have a hard time accepting or advocating the use of NS-2 or any other simulation tool”... * September 2005 archives of the e2e-interest mailing list

17 UW EE Colloquium Feb Questions Do we need to simulate networks anymore? What kind of network simulator is needed? Why should you believe network simulation results?

18 UW EE Colloquium Feb Why simulate? Wireless –Reproducibility –Fidelity (especially, real-time constraints) –Radios may not exist or be available –Field tests in realistic conditions cost $$ Scalability –10,000+ nodes? –For smaller configurations, execution time Simulation is often an appropriate tool for the job

19 UW EE Colloquium Feb Challenges for ns (and simulators)‏ Align with how research is now conducted Improve credibility Can a new simulator help with these problems?

20 UW EE Colloquium Feb What is ns-3? An open source project building a new network simulator to replace ns-2

21 UW EE Colloquium Feb ns-3 project goal Develop a preferred, open simulation environment for networking research 1) a tool aligned with the simulation needs of modern networking research 2) an open-source project that encourages community contribution, peer review, and validation of the software

22 UW EE Colloquium Feb Relationship to ns-2 ns-3 is a new simulator, without backward compatibility Similarities to ns-2: C++ software core GNU GPLv2 licensing ported ns-2 models: random variables, error models, OLSR, Calendar Queue scheduler, (more planned) Differences: Python scripting (or C++ programs) replaces OTcl most of the core rewritten also based on the yans and GTNetS simulators new animators, configuration tools, etc. are in work

23 UW EE Colloquium Feb Funding support ns-3 project is supported by: NSF CISE CRI awards ( ) –PIs: Tom Henderson and Sumit Roy (University of Washington), George Riley (Georgia Institute of Technology) –NSF CNS (University of Washington) INRIA Planete and the French government With additional support from: 2008 Google Summer of Code University of Washington Georgia Institute of Technology ‘

24 UW EE Colloquium Feb Challenges for ns (and simulators)‏ Align with how research is now conducted Improve credibility

25 UW EE Colloquium Feb Test and Evaluation Options When mathematical analysis alone won’t do Increasing realism Increasing complexity Pure simulation Simulation cradles Virtual/Physical testbeds Field experiments Live networks Test and evaluation options Can we develop tools to span this space?

26 UW EE Colloquium Feb Some ns-3 design goals 1.Realism 2.Software reuse 3.Emulation 4.Composition

27 UW EE Colloquium Feb ) Attention to realism Research should often involve a mix of simulations and testbed or live experiments If the simulator cannot be made to closely model a real system: –hard to compare results or validate the model –hard to reuse software between the two domains When in doubt, do what implementations do

28 UW EE Colloquium Feb Attention to realism (example) An ns-3 Node is a husk of a computer to which applications, stacks, and NICs are added Application “DTN”

29 UW EE Colloquium Feb Realism (cont.) Align ns-3 to be more faithful representation of real computers –sockets API –packets are “packed” binary representations ready for direct serialization to/from network devices –alignment with Linux architecture –multiple interfaces handled properly

30 UW EE Colloquium Feb ns-3 follows a Linux architecture class NetDevice class Ipv4Protocol Linux corollaries –each NetDevice is modelled like struct net_device –each layer-3 protocol has a struct in_device component for each NetDevice –this interface (transmit side) is aligned with Linux dev_queue_xmit() –on receive side, demux is modelled after Linux protocol handlers

31 UW EE Colloquium Feb ) software integration Do not reimplement models and tools for which open-source implementations abound ns-3 conforms to standard input/output formats so that other tools can be reused. –e.g., pcap trace output, ns-2 mobility scripts ns-3 is adding support for running implementation code –Network Simulation Cradle integration has met with success: Linux TCP code –ns-3 “process” API

32 UW EE Colloquium Feb ns-3 simulation cradle Port by Florian Westphal of Sam Jansen’s Ph.D. work Figure reference: S. Jansen, Performance, validation and testing with the Network Simulation Cradle. MASCOTS 2006.

33 UW EE Colloquium Feb ns-3 simulation cradle For ns-3: Linux Linux Linux Others: FreeBSD 5 lwip 1.3 OpenBSD 3 Other simulators: ns-2 OmNET++ Figure reference: S. Jansen, Performance, validation and testing with the Network Simulation Cradle. MASCOTS 2006.

34 UW EE Colloquium Feb ns-3 “processes” and Posix API Support for a synchronous Posix socket API each Posix type and function is redefined in the simulator processes get their own private stack –somewhat like a lightweight virtual machine Benefits: –makes porting real world application code much easier –makes writing applications easier because the BSD socket API is faithfully followed see the “mathieu/ns-3-simu” code repository

35 UW EE Colloquium Feb ) emulation support Support moving between simulation and testbeds or live systems A real-time scheduler, and support for two modes of emulation

36 UW EE Colloquium Feb ns-3 emulation modes virtual machine ns-3 virtual machine 1) ns-3 interconnects real or virtual machines real machine ns-3 Testbed real machine ns-3 2) testbeds interconnect ns-3 stacks real machine Various hybrids of the above are possible

37 UW EE Colloquium Feb Example: ORBIT and ns-3 Support for use of Rutgers WINLAB ORBIT radio grid

38 UW EE Colloquium Feb ) Composition ns-3 is a component of an overall workflow Define ns-3 with interfaces to support integration with other tools Topology generators Traffic generators Data analysis frameworks Configuration tools Animators

39 Engineering, Operations & Technology | Boeing Research & TechnologyNetworked Systems Technology Copyright © 2009 Boeing. All rights reserved. Common Open Research Emulator Scalable Network Emulator Network lab “in a box” Efficient and scalable Easy-to-use GUI canvas Kernel-level networking efficiency Reference passing packet sending Runs real binary code No need to modify applications Connects with real networks Hardware-in-the-loop Distributed - runs on multiple servers Virtual nodes process real packets Fork of the IMUNES project University of Zagreb Open Source

40 Engineering, Operations & Technology | Boeing Research & TechnologyNetworked Systems Technology Copyright © 2009 Boeing. All rights reserved. netgraph system Tcl/Tk GUI core_wlan core_span CORE API ng_wlan NIC FreeBSD kernel virtual images (vimages) tunnels userspace Overview of CORE Components Three key components 1. virtual machines 3. graphical user interface 2. networking subsystem

41 Engineering, Operations & Technology | Boeing Research & TechnologyNetworked Systems Technology Copyright © 2009 Boeing. All rights reserved. CORE features IMUNES base: Flexible GUI-based management of FreeBSD lightweight virtual containers Virtual topology can be hooked to real devices CORE extensions: GUI enhancements, kernel enhancements, wireless networking, Linux support (OpenVz and NetEm), distributed emulation, and packaging. Software leads: Jeff Ahrenholz and Claudiu Danilov, Boeing

42 Engineering, Operations & Technology | Boeing Research & TechnologyNetworked Systems Technology Copyright © 2009 Boeing. All rights reserved. CORE and ns-3 integration ns-3 tap-wifi-dumbbell.cc program

43 UW EE Colloquium Feb ns-3 and research alignment In summary, make it easier to move from simulation to emulation to experiments Realism and alignment with popular interfaces Support use of real code Emulation capabilities Combine ns-3 with other tools

44 UW EE Colloquium Feb Challenges for ns (and simulators)‏ Align with how research is now conducted Improve credibility

45 UW EE Colloquium Feb Background on simulation credibility [1] “Why We STILL Don’t Know How to Simulate Networks” –Mostafa Ammar, Georgia Institute of Technology, Annual Simulation Symposium 2005 [2] “Maintaining a Critical Attitude Towards Simulation Results” –Sally Floyd, WNS2 Workshop Keynote, October 2006 [3] “MANET Simulation Studies: The Incredibles” –Kurkowski, Camp, and Colagrosso, ACM Sigmobile, MC2R, Volume 9, Issue 4, October 2005 [4] “An Integrated Approach to Evaluating Simulation Credibility” –Muessig, Laack, and Wrobleski, U.S. Naval Air Warfare Center, August 2001

46 UW EE Colloquium Feb Criteria for Credibility Repeatable Unbiased Realistic Scenarios Statistically Sound Model Accuracy Results Accuracy (Validation) Data Accuracy Usability from [3] from [4]

47 UW EE Colloquium Feb Repeatability A hallmark of the scientific method... Papers should identify simulator, version, operating system, parameters, etc. Better yet, make code and configuration scripts available to the community –Yet, 0 out of 84 ACM Mobihoc MANET simulation papers ( ) referenced publicly available code (from [3])

48 UW EE Colloquium Feb ns-3 approach We are adding support to systematically document how simulations were conducted –A configuration subsystem that dumps simulation parameters to a configuration file (input and output) We will host code, publications, validation results, etc.

49 UW EE Colloquium Feb Model accuracy “error-free-ness” of software and models ns-3 goals here: –Support real code where possible –Open source models –Maintainers

50 UW EE Colloquium Feb Open source models “Given enough eyeballs, all bugs are shallow” –Eric Raymond, “The Cathedral and the Bazaar” ns-3 needs ways to certify models, too –capture level of community acceptance –publication lists, cross-reference –need to identify maintainers, or state the absence of a maintainer –validation techniques and results

51 UW EE Colloquium Feb Example: ns-3 Wifi development Several research groups are maturing the original INRIA model: Karlsruhe Institute of Technology: PHY, e –Equalizing PHY models including capture effects, user-definable coding rates (e.g. 5.9 GHz from p), EDCA QoS extensions of e, Nakagami/Rayleigh propagation loss model University of Florence: n features –Frame Aggregation, Block ACK, HCF (EDCA and support for HCCA),TXOP, HT terminal (also with protection modes), MIMO Russian Academy of Sciences: s –a complete model of IEEE802.11s D2.0 Draft Standard Deutsche Telekom Laboratories in Berlin: PHY Boeing: b channel models, validation (and others...)

52 UW EE Colloquium Feb Pledge break ns-3 needs participation from the research community 1) improving simulation credibility 2) contributed and supported models 3) maintainers Please support your open source projects!

53 UW EE Colloquium Feb Summary of simulation credibility Learn from good and bad examples of simulation research, to produce credible simulations Consider open source (or publishing of models and scripts) to be integral part of your research Please give back to the simulators that you use

54 UW EE Colloquium Feb Acknowledgments Thanks to: the core development team and research project leads –Raj Bhattacharjea, Gustavo Carneiro, Walid Dabbous, Craig Dowell, Joe Kopena, Mathieu Lacage (software lead), George Riley, Sumit Roy 2008 Google Summer of Code mentors and students many code authors and testers the ns-2 PIs and developers for creating ns-2 and for supporting ns-3 activities USC ISI for hosting project mailing lists Contact: (office M354)


Download ppt "UW EE Colloquium Feb. 20091 Network Simulation and Emulation in a Testbed Era Tom Henderson University of Washington The Boeing Company UW EE Colloquium."

Similar presentations


Ads by Google