1. | Los Alamos National Laboratory | April 2013 | UNCLASSIFIED | 1 Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA UNCLASSIFIED Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA Monitoring the Effectiveness of the LANL Operating Experience Program Robert B Stuewe Operating Experience Program Coordinator Los Alamos National Laboratory May 14, 2013

2. | Los Alamos National Laboratory | April 2013 | UNCLASSIFIED | 2 Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA UNCLASSIFIED Scope of the LANL OPEX Program Serves 11,000 people including the Los Alamos Field Office  OPEX Program Stats - 700-800 items processed per year - Multiple pre-planned sources - 50% from DOE (all of you) - All program/function areas - >100 internal institutional per year - 1500-3000 unique users/month - >5000 in searchable archive - >12,000 doc accesses/month - >200 searches/month (up to 400) Customer Portfolio NNSA Weapons Programs NNSA Nonproliferation NNSA Safeguards & Security DOE Environmental Management DOE Energy & Other Programs DOE Office of Science Work for Others ( incl. Nat Security)

3. | Los Alamos National Laboratory | April 2013 | UNCLASSIFIED | 3 Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA UNCLASSIFIED OPEX Monitoring and Evaluation Use Assessments, Measures, Analysis, Stakeholder Feedback  Annual - Assess OPEX Program Implementation and Effectiveness (part of Annual CAS Assurance Process) - Lessons Learned Improvement Workshop (includes Field Office)  Semi-Annually (or more frequently) - Lessons Learned Effectiveness Monitoring (often includes Field Office)  Quarterly - ORPS/Sub-ORPS Trending and Analysis  Monthly - LANL Dashboard  Daily/Weekly - Lessons Learned Archive Analytics  As Occur - Other Internal and External Assessments (3-5 per year) Lessons Learned Images: Adult Mexican Spotted Owl recorded on LANL property in June 2011. In compliance with the LANL Habitat Management Plan, work with heavy equipment must stop in areas of sensitive habit beginning March 1.

4. | Los Alamos National Laboratory | April 2013 | UNCLASSIFIED | 4 Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA UNCLASSIFIED OPEX Implementation Processes are designed to reliably fulfill requirements, and are used by functional, programmatic, and organizational areas important to the site’s mission OPEX Effectiveness “There is objective evidence that experience from operational events is being tracked and used to drive continuous improvement” NNSA NAP-21 Element 2 Effectiveness Criteria Annual CAS Management Self-Assessment  Part of LANS Board of Governors Approved Contractor Assurance System  DOE O210, O 226, and Prime Contract H-4 Clause set primary requirements  Uses feedback from interviews, measures, issues management system, LLEM, and other assessments  Results used by the Annual ISM Verification Lessons Learned in Images: A subcontractor employee at LANL suffered the loss of a fingertip on 2/9/12, when he caught a finger in the bolt hole shown above during the repair of a bucket on a skid-steer loader.

5. | Los Alamos National Laboratory | April 2013 | UNCLASSIFIED | 5 Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA UNCLASSIFIED Monitoring and Evaluation Results Summarizing results from the last two years  The LANL OPEX Program is compliant, implemented, and is demonstrating effectiveness, but opportunities to improve effectiveness still exist  The use of lessons learned for work planning could be improved  Variability is evident in the lessons learned processes used by sub-units  The rate of sharing lessons learned across the Laboratory is at an acceptable level but is trending unfavorably These opportunities for improvement are likely evident across the DOE.

6. | Los Alamos National Laboratory | April 2013 | UNCLASSIFIED | 6 Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA UNCLASSIFIED  Many Lessons Learned are not easy to use for work planning - Too general or truisms (e.g., Always follow procedures and you won’t have problems) - Too site specific (e.g., Procedure GG-XY-0010-1.1 needs to be changed) - Do not include actions that others could use to prevent the adverse event - Does not describe the activity or equipment that was associated with the lesson in a manner that enables quick relevance determination during work planning  Redundancy and confusion in DOE LL directives and processes (When to use which PM lessons learned method, DOE-EM lessons learned process) DOE-Wide Contributing Factors Need to reduce cost-to-share while also creating higher-quality lessons for our activities?

7. | Los Alamos National Laboratory | April 2013 | UNCLASSIFIED | 7 Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA UNCLASSIFIED Additional Details

8. | Los Alamos National Laboratory | April 2013 | UNCLASSIFIED | 8 Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA UNCLASSIFIED Quarterly Lessons Learned Effectiveness Monitoring (LLEM) Lessons Learned in Images: On 8/9/10 at TA-54 Area L, a 4-liter glass container of nitric acid began to vent a red vapor from inside a 14-gallon poly drum. The drum was closed and the area evacuated. No workers were exposed. HAZMAT responded and handled the cleanup. 1.Trending/analysis suggests that an accelerated rate of learning is needed 2.OPEX Program Coordinator selects the focus area, approves the data collection plan 3.OPEX Program personnel go into field to collect data (Field Office Counterpart can and does shadow) Glovebox Safety Lessons Learned Effectiveness Use of LL in Training Development and Delivery R&D Safety Improvement Effectiveness Evaluation Project Management Lessons Learned Procedure Use of LL in Maintenance Work Planning Use of LL in Maintenance Pre-Job Briefs Crane Inspection Card Maintenance LL Follow-up

9. | Los Alamos National Laboratory | April 2013 | UNCLASSIFIED | 9 Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA UNCLASSIFIED OPEX LANL Dashboard Measures Updated Monthly, Action Plans for Red and Yellow levels, accessible by Field Office to support oversight  Processing Time - Urgent within 24 hours - Caution within 7 days  Knowledgebase Inventory  Internal LL Generation Rate  Document Access Rate  Search Rate Staying Green requires monitoring knowledgebase analytics on daily and weekly bases, and taking immediate action before monthly measures suffer! Lessons Learned Images: Plumber’s kit containing propane bottle discovered 3/6/12 at the LANL CMR facility during the performance of an extent of condition immediate action following a TSR violation that had resulted from the discovery of another propane bottle on 2/15/12.