Presentation on theme: "RTCA DO-178C “Software Considerations in Airborne Systems and Equipment Certification” Brock Greenhow March 21, 2013 The main idea of DO-178 is to design."— Presentation transcript:
1 RTCA DO-178C “Software Considerations in Airborne Systems and Equipment Certification” Brock GreenhowMarch 21, 2013The main idea of DO-178 is to design software that meets safety standards, because when you are dealing with software in airplanes certain mistakes could cost people their lives…
2 Software mishaps in Aerospace Engineering Ariane Five rocket explosionSouthern Airways 242Gimli GliderPatriot MissileAriane 5 – 64 bit floating point number relating to the horizontal velocity of the rocket with respect to the platform was converted to a 16 bit signed integer. The number was larger than 32,767, the largest integer storable in a 16 bit signed integer, and thus the conversion failed.Southern Airways 242 – The radar display inaccurately depicted the most severe area of the storm as a calmer area, thus leading the flight crew to take the aircraft into the ensuing hailstorm. In which they were unable to regain thrust in either engine.Gimli Glider – software build that performed measurements and calculations of aviation fuel for this aircraft used metric units (kilograms) whereas the rest of Air Canada's fleet used English units (pounds). Led to running out of gas and eventually having to land elsewhere.Patriot Missile – First the system was only designed to work for 2 hours at a location, and secondly because of the way the Patriot computer performed its calculations and the fact that its registers(4) are only 24 bits long, the conversion of time from an integer to a real number cannot be any more precise than 24 which was not nearly precise enough to intercept the SCUD missileThese are just a few software mishaps in safety critical systems, for the most part the recent historical record for software in aviation has been good, but the future is bringing more risk than ever.
3 Future of Safety Critical Software Increased lines of codeIncreased complexityIncreased criticalityTechnology changesMore with lessIncreased outsourcing and offshoringAttrition of experienced engineersLack of available trainingSome of these are self explanatory:Increased lines of codeIncreased complexityIncreased criticalityTechnology changes: microprocessors can be almost obsolete by the time the product is tested.More with less: Everyone wants to “Go Green” and having less engineers to do the same amount of workIncreased outsourcing and offshoring: market demands and shortage of engineers leads to more of this, which if not overseen properly can lead to issues.Attrition of experienced engineers: current engineers are retiring and without an extensive training program young engineers might not know the reasoning behind key decisions and practices. So they may just do away with old techniques which is not a good idea most of the time.Lack of available training: very few safety focused degree programs, very little formal education on system and software validations and verification
4 Background of DO-178 1982 – DO-178 1985 – DO-178A 1992 – DO-178B 2011 – DO-178C and supplemental material2011 – DO-248C1982 – DO178: very basic information for airborne software1985 – DO178A: strong software engineering principles than DO-178, and has both verification and validation as requirementsalso was not objective based, so hard to assess compliance objectively1992 – DO178B: significantly longer, guidance in form of what (objective) rather than how. Does not include validation of requirements. Required documented evidence for certification.2001 – DO248B: Provides typo and minimal fixes along with FAQ for DO-178B, it is considered clarification not guidance.2011 – DO178C and supplemental material: what the rest of the presentation will be about.2011 – DO248C: same idea as DO-248B but updated for C.
5 Differences from DO-178B to C Added examples and explanationsUsed clearer language and terminologyAdded more objectivesBi-directional tracingParameter Data Item FilesTechnology SupplementsUsed DO-248B to update language and examples to make C easier to understand and more consistent.Added more objectives for levels A, B, and C criticality but I will talk more later about what objectives are and examples.Although it was basically assumed in DO-178B, C makes sure to emphasize this between all requirements, design, code, tests and results.Parameter Data Item Files - Provides separate information that influences the behavior of an executable object code (without changing it). An example would be a configuration file that sets up the schedule and major time frames of a partitioned operating system. The parameter data item file must be verified together with the executable object code, or else it must be tested for all possible ranges of the parameter data items. Added examples and verification of these.Technology supplements are the main differences that will be covered toward the end of the presentation, after you have a better idea of what DO-178C fully entails.
6 ARP4754A System Development System RequirementsAllocate requirements to softwareValidate requirementsCommunicationPlan for changes to come from softwareFor the most part, all that software really needs from the system is a good set of requirements allocated to software. We all have learned what good requirements consist of. When it comes to aviation software there are a few additions like safety, fault correction, timing, and many other.Software also will need the systems engineers to communicate with them well. This helps lead to less problems and confusion. Often times there are issues or ambiguities that come up in systems requirements that need to be clarified in order for software to implement them correctly. Even though the requirements have been validated (have to be in order to be handed off to software!!!) there is almost always changes that will occur, and those changes are assessed and agreed upon by everyone.
7 ARP4761 Aircraft and System Safety Safety Program PlanFHA and SFHA’sPASA and PSSA’sSoftware SafetyImproves with timeErrors are not as obviousNeed specific requirementsInvolve safety and systems in software requirement reviewsThe Safety Program Plan includes a few different assessments. The first assessment is the functional hazard assessment (FHA), which is broken down into system functional hazard assessments (SFHAs) to evaluate what could/would happen if a system were to fail. Then after that assessment comes the Preliminary aircraft safety assessment (PASA) which again is broken down by systems in the preliminary system safety assessment (PSSAs). These use the SFHAs to determine what assurance level is needed for each system. (The tables on next 2 slides will explain more on this)When it comes to just software safety, we all know it can nearly impossible to guarantee correctness on big projects like this and it has been proven that software on this scale gets better as time goes on and more bugs are found and fixed, because it is not always obvious what the errors are and if they cause other problems that are not blatantly seen.So it is smart to include specific software requirements for safety and also a good idea to have some systems and safety people in the reviews of the software requirements to get their input.
8 Safety continued Severity Classification Potential Failure Condition EffectAssurance LevelCatastrophicFailures would result in multiple fatalities and possible complete loss of the airplane.AHazardous/Severe majorFailures would reduce the abilities of airplane or crewmembers to deal with conditions that could result in reduction of safety margins, distress and excessive workload, or even serious or fatal injuries to a small number of people.BMajorFailures would cause similar to issues to the Hazardous/Severe major, but not as severe and likely only injuries and not casualties.CMinorFailures would not significantly reduce airplane safety, and only slight increase of workload and minimal discomfort.DNo safety effectFailures have no effect on the safety of the aircraft.EThis table shows an idea of what the FHA would use to assess a system to determines its severity and thus giving the PASA a clear idea of the assurance levels for the systems and system as a whole.
9 Safety Continued Level Objective Count Objectives with independence E D262C625B6918A7130This table shows how the assurance levels correlate to how many DO-178C objectives and objectives with independence that are needed to get certification for that level of software.
10 Overview of DO-178C Software Planning Software Requirements Software DesignSoftware IntegrationSoftware VerificationSoftware Configuration ManagementSoftware Quality AssuranceSoftware CertificationEach of these life cycle processes will be discussed in detail on the following slides.Certain documents are created in each process that are needed in order for the life cycle to transition to the next process.
11 Software Planning Five Plans Three Standards PSAC SDP SVP SCMP SQAP Software Requirement StandardsSoftware Design StandardsSoftware Coding Standards“If you fail to plan, you plan to fail”(PSAC) Plan for Software Aspects of Certification : contract between applicant and certification authority. High-level overview of the project. Briefly summarizes each of the other plans.(SDP) Software Development Plan : explains the plan for requirements, design, code, and integration. SDP usually references the 3 standards you see below. This plan also lists the type of life cycle model that will be used. It also discusses the environment that will be used for this project. Detailed in the SLECI.(SVP) Software Verification Plan : Plans how the verification of data from SDP will be met. Discusses types of verification including analyses, reviews, and tests. Discusses how independence is met for objectives that need it. Person who coded can’t test it.(SCMP) Software configuration management plan : How the project will use CM to control data produces from SDP and SVP. Major aspect is the pr/cr process of reporting, analyzing, modify, and review changes of baselined data.(SQAP) Software Quality Assurance Plan : Describes how SQ team will assure plans and standards used properly throughout the project, and the major jobs of an SQA engineer.Requirement Standards: give a list of models, rules, tools, and constraints for how to write software requirements.Design Standards: “”””” design software.Coding Standards: “”””” write software code.
12 Software Requirements Foundation to good softwareRefine Systems RequirementsAllocate enough timeSoftware Requirement CycleBi-Directional TracingBaseline SWRD- Just like Building a house with a faulty foundation leads to disaster- Software requirements begin from the refinement and analyses of systems requirements- Problems occur when requirements are not given enough time to be done properly and mature. The pros for taking more time will be great and lead to better requirements that will save money and time in the future, satisfying safety and customer needs, and making testing that much easier.The cycle starts with gathering and analyzing inputs, writing reqs, verifying reqs and back to analyzing if issues come up in verificationWriting reqs involves decomposing into subsystems, prioritizing, classifying reqs (functional or non-functional), establish tracing, and proving feedback to systems.Verifying involves informal peer reviews, and formal reviews with stakeholders and start cycle over if some issues are found.Finish initial cycle and baseline SWRD, transition to Design but allowing all future requirement issues to be fixed via the cr/pr process
13 Software Design Architecture Low-level Requirements SWDD Structural-basedObject-orientedLow-level RequirementsBi-Directional TracingSWDDDesign consists of two components: architecture and designArchitecture: This is broken down into different diagrams depending on the approach of the project, each approach leads to different types of diagrams- Structural based: context diagram, data flow diagram, control flow, decision table, state transition, data dictionary, etc- Object-oriented: use case, state diagram, class diagram, sequenceLow-Level reqs: developed from SWRD and turned into implementation details not needing further info to actually implement them.Ideal design consists of low coupling, high cohesion, modularity, abstraction, robustness, etc like we have learned in classes over the years.Final design becomes known usually as SWDD and once baselined transitions to Implementation.
14 Software Implementation CodingLanguages and compilersGood programmingStandardsTraceabilityIntegrationBuild processLoad processAnalyze memory and addresses- Very important to choose a mature language and compiler when working on safety critical software: 3 main languages used C, Ada, and assembly although other are out there.We have all learned about good programming and the same ideas are encouraged such as readability, reuse, deterministic, error and exception handling, debug proactivelywrite code that complys with the standards referenced in the SDPvery important to establish tracing from source code to SWDDBuild process: compiling and linking the software, needs to be repeatable, relies on controlled environment Software Life Cycle Environment Configuration (SLECI) and needs to be recorded in the SCILoad process: loading the executables into the target computer, also needs to be repeatable and recorded in the SCIAnalyses: need to analyze link data, load data and memory map to ensure no memory is overwritten, addresses are correct, and all components are there.
15 Software Verification ReviewsPlans, requirements, design, test dataAnalysesCode and integrationCoverageOtherTestsRBTs, integrationCases, procedures, resultsTracingVerification accounts for more than half of the objectives, and usually accounts for more than half of the total budget of a project as well.Reviews: qualitatively assess documents like planning documents, requirements, design, code, and test data relating to the test cases, procedures, and results.Analyses: provides repeatable evidence of correctness. There are two main categories of analysis that are needed to comply with DO-178C, and those are code and integration analysis and coverage analysis. Worst Case Execution Time Analysis, Memory Margin Analysis, and Link and Memory Map Analysis are a few of the other types of analysis.Tests: Most tests included multiple test cases for a test procedure. Tests are then executed, and the results are analyzed.Tests need to have normal and robust test cases to fully test requirement. (RBTs)bidirectional tracing between software requirements and test cases, test cases and procedures, and test procedures and results
16 Software Verification Continued Verification of VerificationSCA, MC/DCTest data reviewsProblem ReportingFailures become PR or CRPR or CR processCIASVCPV of V – done after a run of RBTs reviews and analysis.reviewing test procedures, reviewing test results, requirement coverage analysis, structural coverage analysis, statement coverage, decision coverage, and modified condition/decision coverageProblem Reporting – any failure becomes a pr or cr and is then evaluated by all stakeholders to be approved or not.If it is approved, then assigned, analyzed, suggested solutions, agreed upon, implemented, re-verified, re-reviewedSoftware Verification Cases and Procedures (SVCP) – becomes known as the overall test plan for verification, included all tests reviews and analysis.
17 Software Configuration Management Beginning to EndAll life cycle dataCC1 or CC2SCILife cycle data and versionsSLECI and Problem ReportingMaximize productivity minimize mistakesSCM is involved from beginning to endIncorporates basically all of software life cycle data, just for different levels of software some data needs for control than others (CC1 or CC2)The SCI uses the SCM data to provide a list of all life cycle data and their versions in SCMIt is also important for documents like the SLECI to be under SCMAlso problem reporting is important to be under SCM so it can be tracked and understood why certain changes were made or not made.
18 Software Quality Assurance Customer’s needsReview plans and write SQAPLife cycle data audits and approvalReviewsWitness tests, builds, and loadsProblem reportingConformity reviewDocument activities for recordsGoes smoother if quality is stressed at beginning of project and “built in”SQA is involved throughout the project with many different tasks.They review all of the plans including the PSACThey are responsible for auditing life cycle data occasionally throughout the life cycle and if they approve it then the phase transitionsThey sit in on all the different reviews to make sure quality is being built into the projectThey are required to witness some tests and builds and loads to make sure everything is repeatable and done with compliance to plans and SLECIPlay a major role in problem reporting as a part of CRB and track and evaluate all the PRs or CRsPerform the conformity report assurances that software life cycle process are complete, life cycle data is complete, executable code is controlled and can be regenerated.They also document all of their activities throughout the life cycle for SQA records.
19 Software Certification Develop and submit PSACPSAC approvalSubmittal and approval of SCI and SASSOIsThree objectives in DO-178C for certification.To develop and submit the PSACHave the PSAC approved by the certification authorityThird is to submit the SCI and Software Accomplishment Summary (SAS) and have them be approved as well.PSAC and SAS are bookends. PSAC what is going to be done, and SAS is what has been accomplished.Life cycle has 4 different audits during different stages (stage of involvement SOI)- plans and standards have all been reviewed and baselined- at least 50% of the source code has been implemented and reviewed- at least 50% of test cases and procedures have been created and reviewedall issues have been resolved from all previous SOIs and verification results, SCI, and SAS have all been completed, reviewed, and baselinedEach audit encompasses different types of investigations by certification authority based upon what has been completed and if it complies with DO-178C and the plans that were agreed upon.
20 Supplemental Materials DO-330 Software Tool QualificationDO-331 Model-Based Development and VerificationDO-332 Object-Oriented TechnologyDO-333 Formal Methods
21 Software Tool Qualification Separate Document compared to DO-178BThree criteriaTQLLife Cycle similar to whole softwareTool verificationReviewsRBTsMore tools are being used to automate and attempt to speed up certain processes throughout life cycleIn DO-178B it was scattered throughout the document in places where thought tools may be used, but C makes own standalone document.Broken up into 3 criteria: code/file generation (compilers, linkers), verification (tracing, emulation, simulation), and other ones that could also miss an error (more management tools).TQL is based off criteria and what level of software it will be used on ie A, B, C, D, E
22 Model-Based Development and Verification 2 types of ModelsSpecificationDesignBenefitsPotential RisksModels:Specification – high level requirementsDesign – Low level requirementsAlways necessary to have a requirements that are above and external the model to describe the details for enabling model-based development and verificationSimulation is one way of verifying models, but only some objectives can be satisfied that way, others need actual object code created from models loaded into the target.Models need own standard and developed to conform to it.Benefits:Introduced at different levels with different levels of abstraction (system, SWRD, SWDD)Reduce development timeMore focus on requirements ( models present clearer view of system during requirements)Earlier verificationRisks:Difficulties with test completenessTraceability difficultiesSimulation credit controversyModels often mix what and how
23 Object-Oriented Technology Most popularAdditional/Modified objectivesPlansDevelopmentVerificationVulnerability guidanceOO has become extremely popular, almost exclusively taught, there are better tools, and more engineers that know OO.Plans – explanation of virtualization if used, explanation of reused components, and how all OO vulnerabilities will be addressedDevelopment – Tracing is updated that requirements trace to methods and attributes, classes organize the requirements, also specifics of how subclasses traceClass hierarchy for types in high-level requirements, and similarly done for substitution of types in low-level requirements.Verification – activities added to ensure consistency of class hierarchy with the requirements, local type consistency, use of dynamic memory management is robust, also that constructors instantiate objects consistently with requirements.Vulnerabilities:Key features – Inheritance, polymorphism, overloading, type conversion, exception management, dynamic memory management, and virtualizationinheritance - type consistency by verifying that each subclass is substitutable for its superclass. Every verification activity performed on the superclass should also be performed on the subclass.polymorphism, verification activities should show that operations acting on substituted parameters implement the intended semantic behavior. Each unique instantiation of a parameterized type or combination of types should be verified.overloading, the use of explicit type conversion should be used to reduce overloading ambiguity. Verification activities should ensure that type conversions (implicit and explicit) are safe and that all implications are understood.More guidance is in the actual DO-332 and FAQs to clarify how to use the document properly.
24 Formal Methods Changes Benefits Challenges Plans Verification objectivesBenefitsChallengesMathematical proof to verify functionality of a software systemAdditions to plans are needed to explain how formal methods will be used, and what evidence will be providedSVP needs to be updated to ensure no gaps in verification if a combination of formal and procedural methods are used.Some verification objectives are modified or added depending on how formal methods are usedcan replace some review, analyses, and testsimpossible to test executable object code testingIn terms of verification of verification, formal methods can replace all of these objectives.Benefits – improved requirements definition- reduced errors- increased confidence in complex systemsChallenges – lack of education- resource intensive- lack of expertsWith more time formal methods could become used more and be very useful, but they still tend to be confusing and time consuming for most.They do seem to be very beneficial, and are now allowed to be used but I suspect it will be a while before formal methods are mainstream.
25 SourcesPicturesPictures-Question-Mark-Man.jpgInformationRierson, L. (2013). Developing safety-critical software. Boca Raton, FL: CRC Press.Jacklin, S. A. NASA, (2012). Certification of safety-critical software under do-178c and do- 278a . Retrieved from Ames Research Center website:Arnold, D. (2000, August 23). The explosion of the ariane 5. Retrieved fromArnold, D. (2000, August 23). The patriot missile failure. Retrieved fromNelson, W. H. (1997). The gimli glider. Retrieved fromFleury, M. K. (2009, April 29). Crash of southern airways flight 242, georgia. Retrieved from