2 Mission Overview Gene Cernan’s drawings of the lunar sunrise Lunar Atmosphere and Dust Environment Explorer (LADEE) is a NASA mission that will orbit the Moon and its main objective is to characterize the atmosphere and lunar dust environment.Low cost, minimal complexity and rapidly prototyped “common bus” design.Model-Based Software DevelopmentSpecific objectives are:Determine the global density, composition, and time variability of the lunar atmosphere;Confirm the Apollo astronaut sightings of dust jumps and diffuse emissionLaser Communications Demonstration: 622 Mbs Record download rate from the Moon!Gene Cernan’s drawings of the lunar sunriseClementine spacecraft image of moon dust corona
3 Outline Review of LADEE Development Process & Software Architecture Some Lessons LearnedModel Based DevelopmentNPRs/CMMI“Just In Time” organizationsInterface Control DocumentationEmergent BehaviorCurrent Status
4 Flight Software Overview ScopeOnboard Flight Software (Class B)Support Software and Simulators (Class C)Integration of FSW with avionicsGuiding DocumentsNPR Software Engineering RequirementsCMMI Level 2NASA-STD NASA Software Assurance StandardDevelopment ApproachModel Based Development Paradigm (prototyped process using a “Hover Test Vehicle”)5 Incremental Software Builds, 2 Major Releases, 3 final sub-releases5.1: Defects found by I&T and 3DOF5.2: Defects found by Mission Operations Testing5.3: Final RTS set for Golden LoadLeverage Heritage SoftwareGOTS: GSFC OSAL, cFE, cFS, ITOSMOTS: Broad Reach DriversCOTS: , VxWorks, Mathworks Matlab/Simulink & associated toolboxes
5 Model Based Development Iterate Early and OftenRequirementsVerificationDesign/AlgorithmDevelopmentFlight SoftwareModelingHeritageModelsAnalysisVehicle &EnvironmentModelingWorkstationSimulations(eg. Simulink)UnitTestsAutomatedReportingHandDevelopedAppsCodeGenerationHeritageSoftwareIntegratedTestsProcessor-in-the-LoopHardware-in-the-LoopDevelop Models of FSW, Vehicle, and EnvironmentAutomatically generate High-Level Control SoftwareIntegrate with hand-written and heritage software.Iterate while increasing fidelity of tests – Workstation Sim (WSIM), Processor-In-The-Loop (PIL), Hardware-in-the-Loop (HIL)Automated self-documenting tests providing traceability to requirements
7 The Basic QuestionModel-Based Development with significant software re-use: - Hype or Help?Advantages:High level control software in the “Native Language” for GN&C developersAllowed development to be highly parallelizedModular design for all applications, Simulink or hand-developedSoftware developers could prototype autocode/integration process independent of Simulink module developmentEarly Requirements definition led to ability to formalize test infrastructure early in development cycle.Easy to communicate design/algorithms/data flow with stakeholders and other subsystems.Simulink Report generator is an extremely powerful tool for driving verification system.Generated code was generally clean and efficient.Static analysis tool found some defects, which were eliminated through changes in modeling practices.
8 The Basic Question, cont. Disadvantages:Have to be very careful when patching Simulink applications. Must upload associated parameter table. Cannot change interfaces/buses in flight.Bus changes during development induced significant rework of test harnesses.Personal Experience from a Simulink Newbee: LADEE Propulsion ModelSurprisingly fast to model. Amenable to modular development.Think carefully about flow and propagation of signals. Painful to update models/test harnesses.Absolutely needed advice/example models from Simulink expert because of the complexity, range of modeling choices/parameters and their effects on the autocode.Lesson: Model-Based Development is effective when used in a highly disciplined manner. Sloppy development practices lead to bad outcomes, no matter what the language.
9 NASA Procedural Regulations & Capability Maturity Model Integration My Perspective:Daunting set of rules/regulations/practices that have arisen from lessons learned and bad outcomes from other projects.They are an effective roadmap/checklist that help guide on how to plan and document our solutions to prevent those problems.Strategy: Comply as simply and effectively as possible.Lesson Learned:Safety and Mission Assurance/Independent Reviewers can be a real allyProviding templates and advice about effective practices.Extra experienced eyes during the planning phase leads to fewer process issues and software defects later.Early understanding by SMA/IR of project practices leads to smoother and more effective reviews.When you get to the inevitable time crunch at a critical milestone, it is essential to have practiced, documented processes.Multiple examples in LADEE of schedule being brought in by well-practiced test program
10 “Just In Time” Organizations Our “final” 2 build cycles (4 & 5) were major deliveries to the spacecraft.We smugly delivered them “Just In Time” to meet I&T schedule needs.I&T picked up both builds and tested extensively with themI&T and the 3DOF testbed identified ICD related defectsMOS “Just In Time” schedule not tied to these releases.From a FSW perspective, this delayed discovery of:More defects in EDICD/Fault management logic.Hidden Requirements on the spacecraft simulator model.Our concept of “Test Like You Fly” was not the way MOS actually flew.More importantly, the schedule was not tied to “Golden Load” deadlineMilestones for development and certification of MOS RTSs were scheduled for after the need date for inclusion in the “golden load”.Schedule Catch 22: They needed the “golden load” to certify the RTSs for inclusion in the golden load.This put FSW back on the critical path for the spacecraft launch date!Lessons Learned:“Just in Time” philosophy leaves no room for missed schedule dependenciesWe missed valuable test time of the end-to-end operation of the spacecraft leading to a delay in identifying defects. Some were simply too late too be incorporated in the “golden load”
11 Interface Control Documentation The primary cause of defect escape into Build 5.x was misunderstood or ambiguous ICDs.Examples:Fuel Tanks (A,B) = FSW (1,2), Ox Tanks (A,B) = FSW (2,1)Star tracker: Interpreted 8Hz operation to mean heads alternately operating at 4 Hz. Instead, they operated synchronously.Reaction Wheels: “Current Limit Flag” was a warning flag, not an error flag.Mitigation:EDICD in computer readable format and read into database, so could quickly reconfigureAbility to upload parameter tables & software patchesBest prevention for ICD problems was our “Travelling Roadshow”EDU in a mobile chassis loaded with the FSWFlown to payload sites and integrated with instruments.Lessons Learned:Early integration with payloads/instruments essential to clarifying ICDs.Significant rework possible when one “saves money” by not purchasing instrument engineering development units
12 Emergent BehaviorPrior to one of the orbital phasing maneuvers, the spacecraft went into "safemode”Fault Management took action because it detected an excessive spacecraft rate from the state estimation system. It was determined that the star tracker caused a jump in the state estimator signal. Two primary factors:The as-built alignment of the star tracker was slightly different than as-designed.Fixed by a table upload. Reduced rate errors but did not eliminate further safemode transitions.The star tracker was exhibiting delays in providing the spacecraft orientation and position when one of the cameras pointed at a "Big Bright Object" (BBO) such as the Sun, Moon or Earth. The behavior continued to worsen the closer we got to the moon.Reworked state estimator model, reran scenarios, re-verified EST Unit Tests, GN&C L4 Requirements & uploaded patch to the spacecraft.Lesson Learned:The defects we had to correct under schedule pressure and duress in I&T and ORTs were excellent repeated practice for polishing our maintenance processes.
13 Current StatusLADEE is currently in orbit around the moon performing science operations.Successful Laser Communications demonstration: 622Mbs downlink rate. Very useful to be able to download a SDRAM partition in less than 2 minutes.Over 2000 hours of flight.Lowest altitude above the moon’s surface so far is 12.1 KmExpected to operate until mid-April when eclipses will occur.LADEE Flight Software StatusTeam recertified for CMMI level 2 in May 2013Some minor defects found that we’ve worked around.Table uploads performed (ATS, RTS, FM, Thermal updates & defect reduction)2 software patches to account for star tracker behavior1 reboot (still under investigation)LADEE FSW: Enabling a successful mission!