Presentation is loading. Please wait.

Presentation is loading. Please wait.

Sandia National Laboratories is a multi-program laboratory managed and operated by Sandia Corporation, a wholly owned subsidiary of Lockheed Martin Corporation,

Similar presentations

Presentation on theme: "Sandia National Laboratories is a multi-program laboratory managed and operated by Sandia Corporation, a wholly owned subsidiary of Lockheed Martin Corporation,"— Presentation transcript:

1 Sandia National Laboratories is a multi-program laboratory managed and operated by Sandia Corporation, a wholly owned subsidiary of Lockheed Martin Corporation, for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-AC04-94AL85000. Photos placed in horizontal position with even amount of white space between photos and header To replace these boxes with images open the slide master MELCOR 2.1 Software Quality Assurance Program Presented by Larry Humphries 2

2 MELCOR Code Development  MELCOR is developed by:  US Nuclear Regulatory Commission  Division of Systems Analysis and Regulatory Effectiveness  MELCOR Development is also strongly influenced by the participation of many International Partners through the US NRC Cooperative Severe Accident Research Program (CSARP)  Development Contributions – New models  Development Recommendations  Validation MELCOR Code Development New Modeling New/improved modeling Code Performance SQA Validation QA Numerical Stability Utilities SNAP NotePad++ library Uncertainty Tools

3 What is the MELCOR Code  Designed for reactor severe accident and containment DBA simulation  PWR, BWR, HTGR, PWR-SFP, BWR-SFP  Fully Integrated, engineering-level code  Thermal-hydraulic response in the reactor coolant system, reactor cavity, containment, and confinement buildings;  Core heat-up, degradation, and relocation;  Core-concrete attack;  Hydrogen production, transport, and combustion;  Fission product release and transport behavior  Desk-top application  Windows/Linux versions  Relatively fast-running  SNAP for post-processing, visualization, and GUI

4 MELCOR Applications  Forensic analysis of accidents – Fukushima, TMI, PAKS  State-of-the-art Reactor Consequence Analysis- SOARCA  License Amendments  Risk informed regulation  Design Certification  Preliminary Analysis of new designs  Support of International Regulatory Bodies  Non-reactor applications  Leak Path Factor Analysis 5

5 MELCOR Code Development History  MELCOR 1.8.2 (1993)  One of the earliest versions for widespread release.  Not recommended for use  MELCOR 1.8.3 (1994)  BH Package  CORCON-MOD3  Not recommended for use  MELCOR 1.8.4 (1997)  Retention of molten metals behind oxide shells  Vessel creep rupture model  Flow blockage model  Radiant heat transfer between HSs  Hygroscopic aerosols,  chemsorption on surfaces,  SPARC 90  MELCOR 1.8.5 (2000)  CF arguments could be added to plotfile  Consistency checks on COR/CVH volumes  Iterative flow solver added  Diffusion flame model  SS & NS components added for structural modeling  Upward & downward convective & radiative heat transfer from plates  Particulate debris in bypass introduced  Improvements to candling, debris slumping, and conductive, radiative, and candling heat transfer  PAR model was added  CsI added as a default class  Improvements to hygroscopic model  Iodine pool modeling  Carbon steel was added to MP package 6

6 MELCOR Code Development History  MELCOR 1.8.6 (2005)  An option was added to generate input for the MACCS consequences model.  Input was added to simplify conformance with the latest best practices (now defaults in 2.x)  New control functions (LM-CREEP & PIP-STR) for modeling pipe rupture  Modeling of the lower plenum was revised to account for curvature of the lower head  Formation and convection of stratified molten pools  Core periphery model for PWRs to model core baffle/formers and the bypass region  Reflood quench model  Oxidation of B4C poison  Release of AgInCd control poison  Column support structures was added  Interacting materials added to allow modifying enthalpy tables  Spent Fuel Pool modeling  Flashing model  Modified CORSOR Booth release model added  Jet impaction model  Hydrogen chemistry models  MELCOR 2.x (Beta release in 2006)  Code internal structure greatly modified  Dynamic memory allocation  New input format  Formula type control functions  New HTGR modeling (PBR, PMR)  Counter-current flow model  Point kinetics model  Smart restart  Simplified accumulator model  Ability to track radionuclide activities  Turbulent deposition model & bend impaction  Control function for deposition mass for each deposition mechanism.  MELCOR/SNAP interaction in real-time  Full report to user of sensitivity values  Cell-based porosity  Spent fuel pool models  Intermediate heat exchanger /machinery models  Hydrogen chemistry models 7

7 MELCOR Users Worldwide  Argentina  Belgium  Bulgaria  Canada  China  Czech Republic  Finland  France  Germany  India  Italy  Japan  Mexico  Netherlands  Russia  S.Korea  Slovak Republic  South Africa  Spain  Sweden  Switzerland  Turkey  UAE  UK  USA 8

8 DOE Safety Software DOE Safety Software Software Quality Assurance  A self-assessment was performed on September 6, 2007 with the following objectives:  Provide MELCOR software team and management with a perspective of areas of software engineering/quality strength and areas of improvement - relative to the self-assessment instrument and requirements of DOE O 414.1C.  Provide a comparison of the assessment results and gaps for MELCOR with the gap analyses conducted by DOE in 2004.  Provide MELCOR with a potentially sustaining capability to conduct their own self-assessments and manage identified improvement areas.  Sandia Corporate Process Requirement 001.3.6 (CPR 001.3.6)  The software management framework adapted from two internationally recognized standards  the Capability Maturity Model Integration (CMMI) ®  and ISO 9001  These standards provide elements of traceability, repeatability, visibility, accountability, roles and responsibilities, and objective evaluation

9 Software Quality Assurance Annual Re-evaluation Process areas –Project planning and oversight, PPO –Risk Management, RSK –Requirements Development and Management, RDM –Technical Solution, TS –Verification and Validation, VV –Development and Lifecycle Support, DLS –Configuration Management, CM –Measurement and Analysis, MSA –Integrated Product, IPD –Integrated Teaming, ITM –Process Dimensions –Stakeholder Involvement, SI –Ongoing Process Monitoring and Control, PMC –Collected Improvement Data, CD –Objective Evaluations, OEV –Quantitative Objectives Defined for Processes, QPO –Stable Subprocess Performance, SSP –Training, TR –Problem Reporting & Corrective Action, RCA 10

10 MELCOR Software Quality Assurance Best Practices  MELCOR Wiki  Archiving information  Sharing resources (policies, conventions, information, progress) among the development team.  Code Configuration Management (CM)  ‘Subversion’  TortoiseSVN  VisualSVN integrates with Visual Studio (IDE)  Code Review  Code Collaborator  Nightly builds & testing  DEF application used to launch multiple jobs and collect results  HTML report  Regression test report  Regression testing and reporting  More thorough testing for code release  Target bug fixes and new models for testing  Bug tracking and reporting  Bugzilla online  Validation and Assessment calculations  Documentation  Available on Subversion repository with links from wiki  Latest PDF with bookmarks automatically generated from word documents under Subversion control  Links on MELCOR wiki  Sharing of information with users  External web page  MELCOR workshops  Possible user wiki 11 Emphasis is on Automation Affordable solution Consistent solution

11 MELCOR Developers Wiki Site – Internal Use  MELCOR Developers Wiki  Archive records  Requirements, design, and testing  Regression tests  Assessment work  Information Sharing  Debugging Policies  Testing Policies  Code Development practices  Coding Conventions  Lessons Learned  Software Risk Management  Version Changes  Reference Library 12

12 Software Configuration Management (SCM)  Methodology for managing code changes in a team development environment  All SCM systems provide the following essential features:  Concurrency Management  Concurrency refers to the simultaneous editing of a file by more than one person. –Resolve places where code changes conflict  Versioning  Tracks file versions  Makes it possible to roll back changes or recreate a version  Synchronization  Update changes made by other developers 13

13 Subversion (SVN)  Free Open Source  Operates across networks allowing different people access to a central repository  Atomic commits.  No part of a commit takes effect until the entire commit has succeeded.  Revision numbers are per-commit,  Commit's log message is attached to its revision  Branching  Merging branches back into the trunk  Repository can be locked  Especially useful for binary files  Command Line Client  Makes it possible to integrate SVN with other applications  TortoisSVN  Subversion client that integrates SVN with Windows Explorer 14

14 Full Compiler/Linker Options Encoded in MELCOR  Compiler/Linker options printed in output and diagnostic file  Provides a QA check for reproducing results  Is also accessible by running executable with command-line arguments Melcor.exe -config 15

15 Code Review  SQA Requirements:  As outlined in both Sandia Corporate Procedure:IM100.3.2 and ASME NQA-1, an important aspect of a Software Quality Assurance program involves review and documentation for the entire life cycle of software development, from requirements and design to implementation and testing.  Benefits  Code reviews can reduce number of defects in new code  Code reviews can lead to better documentation of code & better understanding of new modeling among team members  Code reviews provide a process by which seasoned programmers can pass experience and knowledge to less experienced programmers.  Improves code readability  Problems  Code reviews can be time consuming or cursory  Code reviews are not well documented  Reviews can be adversarial and not productive 16

16 CodeCollaborator HTML based collaboration –Browser is all that is needed to access –Shared licenses for multiple users Threaded, contextual chat –File changes, chat conversations, and defects are linked together. –Each conversation is threaded by file and line number and can be viewed simultaneously with file content. –Choose between multiple views, or download differences to your local machine for further inspection and testing. –Hyperlink directly to a file or line numbers Version Control Integration –Integrates with Subversion Asynchronous Review –Perform and manage reviews even when participants are separated by many timezones. –Comments are tracked like newsgroups –Review PDFs or Image files 17

17 MELCOR Testing Overview  Unit Testing  All input options should be tested  Ranges of input should be tested.  Values outside of reasonable input should be tested for error messages  Automated Build & Test  Test all revisions and ‘catch’ revisions that break the build  Build on multiple platforms (currently only Windows) and compilers (CVF & Intel)  Suite of fast-running test problems  Objective is to correct problems sooner  MOE utility for searching output for specified test for success criterion  Code Release Testing  Larger suite of test cases  Test recent bug fixes  Test new modeling  Code Test Coverage  Code Profiling 18

18 MELCOR Code Testing  Build Testing  Automated to perform Nightly Builds  MELCOR 1.8.6 Windows Compaq Visual FORTRAN (CVF)  MELCOR 2.X Windows CVF  MELCOR 2.X Windows Intel Visual FORTRAN (IVF)  MELCOR 1.8.6 Linux IVF  MELCOR 2.X Linux IVF  Using CMAKE to generate make files for use on Unix variants to extend building on other platforms & other compilers  Code Testing  Performed Daily (at least frequently)  Standard test cases chosen for physics coverage over 140 test cases  New cases added regularly  Debug & optimized versions tested  Testing of developmental branches  Comparison of results  Consistent results between Windows & Unix  Consistent results between SNL & IBRAE 19

19 Optimization Studies  Optimization can lead to different results  Uninitialized Variables (this is just one possible reason)  Can make it difficult to reproduce errors observed in the release version in the debugger  Optimization Utility to test optimization of each file  Optimized and unoptimized versions of all object files are built  An entire module is tested for optimization first (optimized library is linked with debug library for all other modules)  A fast running test deck is run and the output is compared with the debug version for differences  If differences are observed, each file in the module is tested individually.  Testing of optimized files is performed in parallel  Results indicate that only a few files cannot yet be optimized.  Optimized executables are built with these files unoptimized  Give identical results to debug version !!! 20

20 MELZILLA Bug Reporting  Bugzilla site for bug reporting, tracking, and information  Available from SNL web pageSNL web page  Users submit bugs and details  OS, Hardware, affected packages, severity  Bug description  Attachments  Comments and attachments can be marked private and not visible to other users 21

21 SVN/Bugzilla/Visual Studios/Testing Integration  Integrates Subversion with Windows explorer  All commands are available directly from the windows explorer.  See the status of files directly in the Windows explorer  Allows moving files by right-dragging them in the windows explorer  Integration with issue tracking systems  A separate input box to enter the issue number assigned to the commit, or coloring of the issue number directly in the log message itself  When showing all log messages, an extra column is added with the issue number. You can immediately see to which issue the commit belongs to.  Issue numbers are converted into links which open the web browser directly on the corresponding issue  Optional warning if a commit isn't assigned to an issue number  Integration with MSWord compare  Integration with Visual Studio  All modifications apparent within Visual Studios  Easy to see and check in all modified files (still not automatic) 22

22 MELCOR: Self-Documenting Code  MELCOR generates a complete list of MELCOR Keywords  Global record ‘PrintInputRecords ’  Part of required input processing routine means that all records recognized by MELCOR are printed  MELCOR generates a list of control function arguments recognized by MELCOR  Enabled by ‘PrintInputRecords’  MSWord Macro that scans the user guide document for input records and CF arguments  Comparison with MELCOR list enables identification of undocumented keywords 23

23 MELCOR Quality Assurance: Tracking Code Changes  Changelist  List of code issues and modifications by revision  References to bugzilla site  MELCOR Trends  Provide a very general assessment of code modifications  Code stability  Performance  Metrics –H2 generated, Cs deposition, deposition on filters, CAV ablation  Provided with each public code release  Automated as part of testing 24

24 MELCOR Code Validation  Both Separate Effects and Integral Tests  Part of our regression test suite  Many of these are ongoing analyses  Participation in multiple International Standard Problems  Coverage of most important physics  Heatup/Heat transfer  Oxidation  Reflood  Degradation  Molten pool  FP Release  Vessel failure  Critical Flow  MCCI  DCH  Condensation  Containment stratification  Hydrogen Burn  Hygroscopic effects  Aerosol deposition  RN transport  Iodine pool chemistry  Suppression pool level response  Vent clearing  Engineering Safety Features  Sprays  Ice Condensers  Sensitivity Analysis of Input Analysis  Will be published as Volume III Documentation Report 25 RN Transport FALCON 1 & 2 VANAM-M3 LACE-LA4 LACE-LA1 & LA3 STORM AHMED ABCOVE CSE-A9 DEMONA RTF ISP-41 VERCORS ORNL VI MARVIKEN ATT-4 Containment NUPEC M-8-1, M-8-2 IET 1 through IET7 and IET 9 through IET 11 PNL Ice condenser tests Wisconsin flat plate DEHBI CVTR HDR V44 HDR E-11 NTS-Hydrogen Burn GE Mark-III Suppression Pool Marviken Blowdown Tests CSTF Ice Condenser test LOFT-FP2 COR heatup,degradation, & FP release LOFT-FP2 PBF-SFD CORA-13, Quench 11 DF-4, MP1, MP2 FPT1, FPT3 LHF/OLHF VERCORS ORNL VI Ex-Vessel OECD-MCCI SURC IET-DCH Integral Tests/Accidents Bethsy Flecht-Seaset GE Level Swell RAS MEI NEPTUN TMI-2 FUKUSHIMA

25 Assessment Process After phase III Documents go to editor for final review Review & Approval 26 Phase I (almost complete) Assemble all decks (convert decks if necessary) Collect supplementary documentation Runs calculation to completion Phase II Check that calculation gives reasonable results Run calculation in 2.1 Phase II (ongoing) Update/ initiate assessmentUpdate spreadsheet Presentation at weekly assessment meeting Commit all files (decks, XLS, Word) to repository Transfer test case to another analyst for Phase III review Assessment should be complete Phase III (ongoing) Second review (presentation if necessary) Re-run with final code version Clean up input deckDraft of write-up

26 MELCOR User Workshops and Meetings  MELCOR Workshop  September 12-15, 2011  More than 70 participants  Use of SNAP  September 10, 2012  MELCOR Code Assessment Program (MCAP)  September 22-23, 2011  September 13-14, 2012  European MELCOR User Group (EMUG)  ENEA: April 11-12, 2011  GRS : April 16-17, 2012 27 EMUG 1 EMUG 2 EMUG 3 EMUG 4

27 Questions? 28

Download ppt "Sandia National Laboratories is a multi-program laboratory managed and operated by Sandia Corporation, a wholly owned subsidiary of Lockheed Martin Corporation,"

Similar presentations

Ads by Google