1. July 30, 2003SAIC @ NASA Glenn Research Center1 Programmable Logic Devices Building the Case for Software-style Assurance Kalynnda Berens Kalynnda.Berens@grc.nasa.gov

2. July 30, 2003SAIC @ NASA Glenn Research Center2 What is Programmable Logic Programmable Logic Controllers (PLC) Programmable Logic Devices –Field Programmable Gate Array (FPGA) –Application Specific Integrated Circuit (ASIC) –System-on-chip (SOC) –Complex PLD (CPLD) –others

3. July 30, 2003SAIC @ NASA Glenn Research Center3 The Hardware/Software Boundary Software BIOS/bootstrap Operating system Applications Programmed Easily changed Can “do anything” Cannot be 100%, exhaustively tested Firmware Software residing in non- volatile storage Electronic Hardware ICs Microprocessor A/D, D/A Sensors Off-the-shelf components Exhaustively Tested by Vendor Programmable Logic Controllers Special purpose computer (process control) Uses LadderLogic, other languages for programming SOC Reconfig. Computing Programmable Logic Devices FPGA CPLD PAL ASIC Designed with HDL Compiled/Programmed May be reprogrammable in the field Cannot be 100%, exhaustively tested

4. July 30, 2003SAIC @ NASA Glenn Research Center4 Pushing the Limits System-on-Chip (SOC) –Combine microprocessor/input/output, often FPGA for programmability Reconfigurable Computing –Morphware, Configware, Flowware –In NASA Strategic Technology Plan FPGAs –30,000 to over a million gates –Complex interactions

5. July 30, 2003SAIC @ NASA Glenn Research Center5 Complexity Types of faults –Incomplete specifications –Design and Implementation Errors (Common mode) –Unexpected or unanticipated combinations of valid operating states. –Unintended interactions –Unknown defects in tools (design or verification)

6. July 30, 2003SAIC @ NASA Glenn Research Center6 Hardware/Software Differences Most PL cannot be changed once “burned” (programmed). FPGAs can be programmed on-the-fly. Software execution is serial – one instruction after another PL execution is parallel – multiple simultaneous signals and processes PL designed, verified, tested by engineers

7. July 30, 2003SAIC @ NASA Glenn Research Center7 Assurance: Product and Process ActivityProductProcessEng.QA Requirements Specification XXR Design Documentation XXR Requirements, Design Analyses XX Inspections, Walkthroughs XXX Simulation XXW Testing XXW Planning (Risk, Management, Development, QA) XXX Configuration Management XXX Audits XX

8. July 30, 2003SAIC @ NASA Glenn Research Center8 Current PL Process Design from system requirements Functional Simulation –Includes “corner cases” Testing (unit and system) –Simulation and unit test usually performed by design engineer May perform code coverage measurement Verification takes 70% of design task

9. July 30, 2003SAIC @ NASA Glenn Research Center9 NASA PL Assurance Activities – from the user’s point of view YesNo Review source1531 Witness Programming938 Witness Testing1631 Verify Version1234 Audit development1134 Audit CM1632

10. July 30, 2003SAIC @ NASA Glenn Research Center10 NASA PL Assurance Activities – from QA’s point of view ProjectSAOther QAOtherNone PL Testing911 Test Witness6232 Code Review5121 2 Witness Burn6131 3 CM Audit1415 Devel. Audit235 FCA235 PCA1315 VDD2216 Safety Verif.8443232 1 Vendor or contractor 2 Safety personnel

11. July 30, 2003SAIC @ NASA Glenn Research Center11 What are others doing? Hardware/software co-verification Industry/Military practices still open issue – tough nut to crack ESA – starting to address FPGA/ASIC through reports and guidance FAA – DO-254 for Complex Electronic Hardware, calls for design process assurance

12. July 30, 2003SAIC @ NASA Glenn Research Center12 PL-related Standards and Guidelines IEC-61508 - “Functional Safety of Electrical/Electronic/ Programmable Electronic Safety-related Systems” DO-254 – “Design Assurance Guidelines for Airborne Electronic Hardware” IEC-1131-3 – “PLC Programming Languages” IEC-61511 – “ Functional Safety - Safety Instrumented Systems For The Process Industry Sector ” IEE SEMSPLC – “ Software Engineering Methods for Safe Programmable Logic Controllers ”

13. July 30, 2003SAIC @ NASA Glenn Research Center13 PL-related Standards and Guidelines (continued) European Space Agency –Space Product Assurance – ASIC Development –VHDL Modeling Guidelines –FPGA report EWICS TC7 Guidelines for the use of Programmable Logic Controllers in Safety-related Systems

14. July 30, 2003SAIC @ NASA Glenn Research Center14 FAA and DO-254 Complex Electronic Hardware includes FPGA, CPLD, and ASIC DO-254 required for Levels A and B (highest criticality) Defines Hardware Life-Cycle Processes: –Planning Process –Hardware Design Processes Requirements, Design, Implementation, Production, Test –Validation Process –Verification Process –Configuration Management Process –Process Assurance –Certification Liaison Process

15. July 30, 2003SAIC @ NASA Glenn Research Center15 DO-254 at Langley Case study on applying DO-254 to SPIDER 1 Implemented process assurance –monitoring the development activities to assure they are in accordance to plans Placed conceptual design in CM Used Formal Methods 1 Scalable Processor-Independent Design for Electromagnetic Resilience

16. July 30, 2003SAIC @ NASA Glenn Research Center16 FPGA Lessons Learned European Space Agency, 2002 Reviewed FPGA’s in ESA/NASA missions –Extensive use in critical systems with little thought to SEU –Design and verification by same individual –Insufficient verification due to inadequate stimuli selection –Test only – simulation often skipped –Non-engineers “blessing” design FPGA’s are “the software of the hardware world” –Encourage engineers to quickly get to hardware test, ignoring good design practice

17. July 30, 2003SAIC @ NASA Glenn Research Center17 Safety-Related Complex Electronic Systems, 2000 Simulation alone is not adequate. Exhaustive list of possible failures not possible. –Strengthen system/subsystem tests –Consider origin of faults Errors of specification, design, production Internal faults External faults Quality of vendor-supplied soft core or macro libraries is not guaranteed Synthesis tools can generate faults High fault coverage in test is mandatory

18. July 30, 2003SAIC @ NASA Glenn Research Center18 What do we know? PLCs use software, just the purpose and language differ FPGAs and other Programmable Logic devices are very complex Process assurance provides additional value in conjunction with product assurance Process assurance currently not applied to most PLD development

19. July 30, 2003SAIC @ NASA Glenn Research Center19 What don’t we know? Industry and Military QA Best Practices What level of process assurance is required Who should do QA on programmable logic –Hardware QA More likely to understand PL or quickly learn Need to learn process assurance activities –Software QA Familiar with process assurance Would need to learn PL hardware and language How to integrate process assurance in NASA –Software CMM implementation may provide guide

20. July 30, 2003SAIC @ NASA Glenn Research Center20 Software->Hardware Assurance Inspection of HDL code and schematics –Validated as low-cost, high-probability of catching errors for hardware Walkthroughs Independent test team Formal Methods Complexity measurements Traceability Change impact analysis CM tools and processes Functional, code coverage analysis QA monitoring of development process

21. July 30, 2003SAIC @ NASA Glenn Research Center21 Hardware->Software Assurance Simulation, test beds are standard operating procedure Testing against boundary conditions (“corner cases”) Wide variety of available tools for verification

22. July 30, 2003SAIC @ NASA Glenn Research Center22 Next steps Goal is not to provide the answers to how PL is assured, but to set the parameters for constructive discussion within NASA and provide a common information base –Issue Paper on this topic –Process Assurance guidance for Hw QA –PL/Hardware guidance for Sw QA

23. July 30, 2003SAIC @ NASA Glenn Research Center23 Please Take the Survey! http://osat-ext.grc.nasa.gov/rmo/plcsurvey If you have industry/military QA or engineering contacts, please email me at: Kalynnda.Berens@grc.nasa.gov