1. National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager

2. Agenda Eldec Test Engineering Priorities V I Engineering Overview Common Software Engineering Issues Discussion

3. Eldec Priorities What are the top initiatives within the test engineering group? How do you currently measure success/failure? What issues does your group currently face? Are your suppliers required to follow process/development guidelines?

4. History of V I Engineering Founded in 1992 Select NI Alliance Member Four Locations –Farmington Hills, MI (HQ) –St. Paul, MN –Indianapolis, IN –Los Angeles, CA Business Units –Engineering Services –Engineering Information Management Systems (EIM) –VISTA

5. VIE – Strategic Model Core Competencies Business Units Partners in Test Solutions Engineering Services Engineering Information Management VISTA Project Management Software Expertise* *Most LabVIEW and TestStand certified developers and architects in the world in 2004

6. Multi-Industry Experience

7. Microsoft – EOL XBOX Controller Testers

8. LLNL – Pathogen Detector

9. Ford/GM – Safety System Automate Crash Tests Common Database and Reports

10. VISTA Software engineering approach to the management, development and release of LabVIEW and TestStand software. Process Consulting Training Development and Process Tools

11. Process Implementation Process Analysis and Definition –Prioritize improvement initiatives –Define development processdevelopment process –Develop templates and guides –Select automation tools Implement Automation Tools Training –Software development process –Development and process improvement tools –Common programming techniques and architecture

12. VISTA Tools Configuration Management –Basic CM, File Load Check, Baselines, Deployment with Integrity Project Management –Metrics Tracking (Time, Nodes, GOBs, Complexity) Quality Assurance –Requirements Traceability, Statement Coverage Reuse –Documentation Tools, Style Guide, Reuse Library GOOP –GOOP Inheritance Toolkit, UML Editor

13. Benefits Decreased Development Cost Decreased Software Maintenance Increased Quality Traceability Improved Decision Making with Metrics Industry Certification “VISTA expanded my horizons. We are moving forward with VISTA because it promotes better programming practices and will raise our quality.” Keith Burris – Lawrence Livermore National Laboratory

14. VIE’s Value Proposition Focus on Core Competencies –LabVIEW & TestStand –Project Management Continuous Improvement with VISTA Awareness of Business Requirements (DoD, CMMI) Advanced S/W Design (GOOP, UML, SyML) Establish Partnerships –Hardware –Local support

15. Simplified Waterfall Model for Software Design research/ definition requirements specification designcodingtesting ship/ maintain

16. research/ definition requirements specification designcodingtesting ship/ maintain All Too Often… LabVIEW makes it very easy to quickly put something together Developers do not follow a process

17. Case Study Source: CMU/SEI-94-TR-013

18. Case Study Source: CMU/SEI-94-TR-013

19. Common Software Engineering Issues Process Issues –Configuration Management –Project Management –Requirements Management –Quality Assurance (V&V) Development Issues –Scalability –Reusability This leads to poor code quality and inefficient use of resources

20. Addressing the Issues SEI (Software Engineering Institute www.sei.cmu.edu)www.sei.cmu.edu –SW-CMM –CMMI IEEE Software Engineering Standards (www.standards.ieee.org/software)www.standards.ieee.org/software –828, 830, 1028, 1220,1058.1 ISO/IEC –9001:2000, 12207, 15288 Industry Guidelines –FDA –DoE –FAA

21. Process Implementation Process Analysis and Definition –Prioritize improvement initiatives –Define development processdevelopment process –Develop templates and guides –Select automation tools Implement Automation Tools Training –Software development process –Development and process improvement tools –Common programming techniques and architecture

22. Challenges with Process Implementation for LabVIEW and TestStand LabVIEW File Handling –Embedded Source and Object –Static Linkage Metrics –Complexity –Size Design Methodology –Style –Patterns

23. Configuration Management

24. Why use Configuration Management? –To know what you’ve built (configuration) –To identify and track needed changes –To track status of configuration items –To deliver known and verified configurations Four main Configuration Management Functions –Configuration identification –Configuration control –Configuration status accounting –Configuration authentication

25. CM Terminology SCC – Source Code Control Configuration Item – files placed in SCC Baseline – specific versions of CI’s at a specific time in the product’s lifecycle (ie Beta Release, Rev 2.0.1) Status Accounting – knowing which version belongs in a baseline and why Label – assigned as an attribute to a specific version of a file or application, may be used to show status accounting (Alpha, Beta, Reviewed, Verified, etc.)

26. Example CM Plan and ProcessCM Plan Process Configuration Items (specifications, code, daq settings, support files, exe’s) Configuration Control –User roles (developer, lead developer, project manager, operations) –Permissions (read/write access, labels) –Change proceduresChange procedures Status Accounting –Labeling procedures Authentication –Audits –Integrity checks

27. CM Recommendations Treat read only VIs as locked Establish common development environment Do not allow multiple checkouts (merging impossible) Avoid branching SCC projects (cross linking and merging) Do not use LLBs during development (can not locate modifications within) Use labels during development (assign status) Deploy exe or use integrity check

28. CM Integration Solutions - demos Basic –LV 6.0.2 PDS built in SCC –SCC repository independent of LV Intermediate –LV 6.0.2 PDS and newer built in SCC with VSS or Perforce Advanced –VISTA CM Tools for LV 6.1 (Base, Full, PDS) and newer with VSS or ClearCase (others in development)

29. Importance of Metrics Confirm Estimates Project Monitoring Cost Justification Risk Analysis Impact of Change Track Improvement Initiatives –Reuse –Productivity –Diagnose root cause of project failure “You can’t manage what you can’t measure”

30. Project Management

31. Standard Metrics (text based languages) Hours SLOC Function Points Cyclomatic Complexity EVA (Earned Value Analysis)

32. LabVIEW Metrics Hours SLOC Equivalent Metric –Nodes (VI Metrics) –GOBs (Graphical OBjects calculator) Cyclomatic Complexity (possible, but not yet available) EVA

33. TestStand Metrics Hours SLOC Equivalent Metric –Lines in seq file EVA

34. Metrics Trials LabVIEW Developer Exams –GOBs more reliable than nodes V I Engineering –~10 GOBs/hr for product development and regulated industries –~25 GOBs/hr for automotive solutions Ball Aerospace –8-10 GOBs/hr for 2 week to 2 man year projects General Dynamics (CMMI Level 3) –8.7 GOBs/hr avg for 8000 man hours of work

35. Variables Style Design Patterns / Architecture used (GOOP vs event driven vs traditional) Quality Level Requirement –Industry (regulated vs non-regulated) –Application Type (product, manufacturing test system, DVT, etc.)

36. Establishing a Project Plan with Metrics Review Requirements Decompose Requirements into Tasks/Components –Complete preliminary design –MS Project example WBSMS Project Estimate Development Tasks based on Historical Metrics and Risk (spreadsheet example)spreadsheet example Select Lifecycle Identify Quality Goals Assign Estimate to Non-development Tasks based on Historical Metrics based on Quality Goals and Lifecycle

37. Establishing Project Control Identify Project Deliverables for each SLM Phase Build Master Project Schedule Assign Effort-Based Planned Value to Deliverables with Project Schedule –SW deliverables back-fire GOBs from hours estimates

38. Project Monitoring with Metrics Track % complete for Deliverables –Use GOBs, SLM phase, status on tasks for objective feedback on software deliverables Roll Up % Complete to EV Track EV vs. Planned Value, Actual Cost Manage by Budget at Completion vs. Estimate at Completion Variance, etc.

39. Project Monitoring with Metrics Track % complete for Deliverables –Use GOBs, SLM phase for objective feedback on software deliverablesGOBsSLM phase Roll Up % Complete to EV Track EV vs. Planned Value, Actual Cost Manage by Budget at Completion vs. Estimate at Completion Variance, etc.

40. Project Monitoring with Metrics Track % complete for Deliverables –Use GOBs, SLM phase for objective feedback on software deliverablesGOBsSLM phase Roll Up % Complete to EV Track EV vs. Planned Value, Actual Cost Manage by Budget at Completion vs. Estimate at Completion Variance, etc.

41. Reuse

42. 42 Coding for Reuse in LabVIEW Benefits of LabVIEW Reuse What is Required for Effective Reuse with LabVIEW? Reuse Tips

43. Benefits of LabVIEW Reuse Decreased Development Time Reduced Risk Improved Quality Lower Overall Cost

44. Requirements for Reuse Reuse Development Philosophy –Reuse must be a part of the organizational culture Coding Standards / Style GuidesStyle Guides –Coding style needs to be familiar to entire team Training Source Code Control

45. 45 Reuse Development Philosophy Always Evaluate Reuse Potential Generalize Code; Avoid project specific code Document for Reuse and Maintenance Have Reuse Reviews at the Beginning and the End of the Project

46. CM and Reuse Avoid VIs with the Same Name Put LabVIEW Reuse Libraries in user.lib (treat them similar to vi.lib) –Organize VIs based on code categories –Limit check-out/check-in access to reuse librarians Categorize LabVIEW Libraries –General reuse, product line reuse, product specific Maintain Backward Compatibility –Create a new VI before losing backward compatibility

47. 47 Example Reuse Library Directory Structure in user.lib..\VIE Library.lib\ArrayTools.llb..\VIE Library.lib\ConfigUtilities.llb..\VIE Library.lib\ControlFunctions.llb..\VIE Library.lib\Dialogs.llb..\VIE Library.lib\ErrorTools.llb..\VIE Library.lib\FileUtilities.llb..\VIE Library.lib\GUI.llb..\VIE Library.lib\GUI.llb\Soft Keyboard.llb..\VIE Library.lib\NumericTools.llb..\VIE Library.lib\OS Utilities..\VIE Library.lib\OS Utilities\Windows.llb..\VIE Library.lib\SharedTYPE.llb..\VIE Library.lib\Sound.llb..\VIE Library.lib\StateMachine.llb..\VIE Library.lib\StringTools.llb..\VIE Library.lib\Time.llb..\VIE Module.lib\AI and Calibration.llb..\VIE Module.lib\DTS Engine.llb..\VIE Module.lib\Formula Evaluator.llb..\VIE Module.lib\SecurityModule.llb..\VIE Module.lib\System Integrity Module.llb (.llb extensions are used for directory names to be consistent if converted to VI library files)

48. 48 LabVIEW Style Affects Reuse Using a common style and good VI documentation can significantly improve reuse LabVIEW Development Guidelines, chapter 6 –devstyle.pdf in..LabVIEW\Search the LabVIEW Bookshelf… Use a code review checklist Avoid overly complex code Use templates to help standardize coding style between multiple developers

49. 49 Architecture Reuse Tips Use layered architectures; limit interaction to adjacent layers Avoid project specific dependencies; use loose coupling –Use Graphical Object-oriented Programming (GOOP) –Avoid global variables –Use standard configuration tools –Separate user-interface VIs from the main application Think in terms of reusable components –Encapsulate functionality and data (use GOOP) –Remember that how features specific to an architecture are used may limit reuse to that architecture.

50. VISTA Software engineering approach to the management, development and release of LabVIEW and TestStand software. Process Consulting Training Development and Process Tools

51. VISTA Configuration and Project Management Tool Workspace Metrics CM Functionality –LV and TS integration –Baselines –Build with integrity (exe or native) –File comparison –File load check

52. VISTA Productivity and Reuse Tools Help Generator Renaming and Relocation Tool Documentation Tool Reuse Library Style Guide

53. VISTA Quality Assurance Node/Path Tracker Cyclomatic Complexity Traceability Tool

54. VISTA GOOP Tools GOOP Inheritance Toolkit –GOOP Wizard 3 with Inheritance –Icon Editor –Development Distribution Tool

55. GOOP Background Scalability – this non-OOP is not easy to grow or maintain

56. GOOP Background Scalability – layered, OOP approach

57. How Can V I Engineering Help You? Process Assessment Custom and Advanced Training Tools Project Support Architecture Design Learn more at www.vista.viengineering.com www.vista.viengineering.com