1. Learning About Learning Progress in Lessons Learned at Los Alamos National Laboratory presented to the DOE Operating Experience Coordinators Workshop May 5, 2010
Bob Stuewe
Operating Experience Coordinator
Contractor Assurance Office – Performance Feedback
Los Alamos National Laboratory

2. Bottom Line Results Take About Three Years to Manifest
End of CY 2006
FY 2010 (Now)
163 processed per year
Pass preliminary screening
Nearly all DOE/HSS
Safety
10 internal per year
200 web users/month
250 in searchable archive
?? document accesses per month
?? searches/month
Trend Occurrences
Track Communications
2 FTEs
1 FTE on the Trending
800 processed per year
Pass preliminary screening
Multiple pre-planned sources
All program/function areas
internal per year
web users/month
>2100 in searchable archive
>30,000 document accesses per month
searches/month
Analyze/Trend Occurrences, Causes
Track Actions (risk-based)
2.5 FTEs
1.2 FTE on Analysis

3. Learning About Learning Lessons About Effective Lessons Learned Programs
Champion Learning, Not Lessons
Reliable Processes Build Trust and Use
Clarify Roles to Eliminate Redundancy
Support Work Processes
Enable Risk Management
More Focused Equals More Effective
Build and Use Indicators of Effectiveness
Encourage Innovation

4. Champion Learning Not Lessons Operating Experience Programs are Key to Knowledge Management
World of Possible Knowledge
Operating Experience Program
Learning happens in the work unit in many, many ways
(Required reading, training, requirements updates, work team meetings, plan of the day, procedure updates, learning teams, process improvement teams, corrective action teams etc.)

5. Champion Learning Not Lessons From the Criteria for Performance Excellence (MBNQA)
Learning is one of the three pillars of organizational performance.
Learning is referenced 103 times in 88 pages
“The term “performance excellence” refers to an integrated approach to organizational performance management that results in (1) delivery of ever-improving value to customers and stakeholders, contributing to organizational sustainability; (2) improvement of overall organizational effectiveness and capabilities; and (3) organizational and personal learning.”
“To be effective, learning should be embedded in the way an organization operates.”

6. Champion Learning Not Lessons Know How Your Organization Learns and Support the Methods
Nuclear and High Hazard Facilities
Conduct of Operations: required reading, procedures, FLM weekly meeting, critiques, facility lessons learned coordinators, etc.
Subject Areas
Electrical Safety Committee, Chemical Safety Committee, Glovebox Safety Committee, Security Incident Analysis, etc.
Lower Hazard Organizations
Worker Safety and Security Teams, Learning Teams, Management Meetings, mentoring, on-boarding, etc.
Institutional
Management review boards, requirements updates, training updates, institutional web pages, etc.

7. Reliable Processes Build Trust and Use Trust Requires Relationships – With Products/Services Used
Operating Experience and Lessons Learned Process
Collects, screens, risk-prioritizes, and distributes operating experiences and lessons learned from key external and internal sources using timely communications.
Three risk-priority levels with different disposition/communication pathways
Lessons Learned and Operating Experience Archive
Captures relevant lessons and best practices reported by the Laboratory managers and workers, contractors, subcontractors, and from other DOE sites and external sources.
Knowledge base
Mirror: Translating Events into Actionable Information
Communicates high-leverage actions, lessons learned and best practices associated with operating experience patterns and trends.

8. Reliable Processes Build Trust and Use Process Management is Proven To Create Reliable Products/Services
Establish Process Management
Lean Six Sigma - Champion and Yellow Belt
Process Team (Weekly - Operations)
Integration Team (at least Annually - Integration and Strategy)
Benchmark (inside and outside DOE)
Define Product/Service Quality Specifications
Purpose -> Products/Services-> Customers-> Requirements -> Controlling characteristics
Defined processes (steps and roles) for each product (mapping)
Set up Process Controls and Improvement
Program Description, Procedures, Checklists, Tools
Use Contractor Assurance System: Goals, Measures, Assessments, Issues Mgt, Process Improvement
Establish Task Roles (daily, weekly, monthly, quarterly, annual)
Annual Improvement Plan (effectiveness and efficiency)
Execute and Innovate

9. Reliable Processes Build Trust and Use Reliable Processes Requires Reliable Operations
Monday
Process Team
Updates
Upcoming
Communications Plan
Operations
Improvement Plan
Tuesday
Integration
CAS Management Team
Institutional Management Review Board
As Needed
Wednesday
MIRROR Team
Organized by sections of the publication
Quality review and improve key messages and actions
Design year-end and next issue
Thursday
Screening Team
Process Team and occasionally the site office
Finalize screening
Determine dispositions
Write headlines
Publish Weekly Summary
Friday
Week Close Out
In case extra time is needed to complete the weeks deliverables
Backlogs are deadly
Daily intake and preliminary screening
Work on products and services (MIRROR, lessons, support customers)
Work on Communications Plan and Improvements Plan

10. Clarify Roles to Eliminate Redundancy Functional Approaches Increased Costs, Decreased Effectiveness
LL requirements embedded in many directives, guides
Alignment and Integration can save taxpayer money
LANL approach
OPEX program supports multiple functions – use an integration team to plan annual improvement (Safety, Security, Env Mgt, Project Mgt, Facilities, HPI, Training)
Issue & Corrective Action Management - handles extent of condition for institutional/local, OPEX program helps with what’s in between
QA/SCI – handles work-related suspect, counterfeit and defective items, OPEX serves as knowledge base, get directly involved at Urgent level
Planned integration with internal security LL and Security Lessons Learned Center
Organizational and Subject Area LL Knowledge Bases and Process align with and link to Institutional
Contractor Assurance System: Measures, Assessment Planning, Issue Effectiveness Evaluation
Site office uses LANL screening and disposition process, oversight of higher risk

11. Support the Work Processes Easy to Think About the “Push”

12. Support the Work Processes Work is Always Underway – Don’t Try to Change Cars While Moving

13. Support the Work Processes Beware of the Adverse Unintended Consequences of the “Push”
Common Lesson from OPEX: Not Thinking Through a Change
Nuclear Facilities
Procedures control the quality
Modifying procedures include many steps, for good reason
Conservation of Attention Law
Only so much time in a day
Rushing to implement a lessons learned can create rework
Resource Allocation is Not an OPEX Role
Acting on a lesson competes for resources
OPEX Coordinators do not know the daily priorities on the work floor

14. Support the Work Processes Hard to Think about the “Pull” – But Equally Important

15. Support the Work Processes Supporting Work Processes Requires a Knowledge Base
MS Sharepoint
Reduced Cost
IT Costs
Process Costs
Aligns with Work Processes
Topical Libraries (key systems)
Alerts (individual and topical groups)
How-To Tools
Full PDF search capability

16. Enable Risk Management To Engage Senior Management – Help Reduce Risk
Hazard inventory is pretty stable at the sites
Sharing known or low hazards provides little value to management
“New” and “Significant” Hazards important and valued
Risk Management
About reducing consequence or probability
What is the risk? (to the mission)
How large are the potential adverse consequences?
What are you doing to handle it?
How likely are you to succeed?
Should you do more?
Requires knowledge of organizational processes
Russian Hydro as an Example
Where Lessons Learned can add the most value to management.

17. Enable Risk Management Risk-based Prioritization Matrix derived from O210.2 with Security from IMI levels Urgent and Caution matrix essentially identical to Order – will add Best Practice un future
Urgent = Immediate action is needed to ensure that the Laboratory is in a safe, secure, and environmentally sound mode relative to the information identified in the alert. Active engagement of relevant manager(s). Formal issues and corrective action tracking.
Work Planning
Processes (Line)
Training
Development (Support)
Requirements
URGENT
Monitor
Sources
Screen
Clarify
Communicate
Caution
Informational
Caution = Timely action may be needed. Routine notification through Lessons Learned and Operating Experience Weekly Summary s and LANL Home Page. LANL managers determine relevance and take needed action.
Informational = Future action may be needed. Posted to LANL Home Page if broadly applicable and actions clear. Important to knowledge base.

18. Enable Risk Management Risk Levels Differ from Site to Site
Pre-Screening
Post Screening #
HSS - Green
Informational 21
HSS - Blue
Caution 4 44
HSS - Red 6
HSS - Yellow 38 65
Urgent Lessons most commonly associated with DOE Safety Alerts and Advisories, defective safety items, and other sources

19. Enable Risk Management Select Sources to Support Risk Management – Might Change over Time
Primary Routine Sources
Monitoring
Frequency
DOE HSS Server/Web (LL, Alerts, OE Summaries, Security Lessons Learned Center (SLLC))
Daily
Partner Alerts (Bechtel, BWXT, UC, etc.)
As Issued
ORPS (from Other Sites)
Suspect Counterfeit/Defective Items
U.S. Chemical Safety Board
Monthly
Enforcement Actions and GAO Reports
Quarterly
Consumer Products (CSPC, FDA)

20. More Focused Equals More Effective Tailor Communication Methods to Audience and Media Capability

21. More Focused Equals More Effective Weekly Summary – Management and LL Roles with Prioritization
Recipients
Division Leaders and above
NNSA Site Office
LL Roles and Upon Request
Augments daily ORPS
Specific = Relevant
Define Most Specific Customer
Select Words in Headline
Integrity Check
Order (First and Last Most Read)
Scan-able (6 most important/actionable; > 6 has lower read rate)

22. More Focused Equals More Effective LANL Homepage to Reach Workers Directly
Some Communication Overlap Helps Effectiveness
Main Users
Workers
Broadest lessons
Includes worker home safety
Specific = Relevant
Even shorter headlines
Pre-job brief is major customers

23. More Focused Equals More Effective Internal Lessons Learned Submission – Content over Format
Adobe Fill-able Form
Just the Facts
Word Docs too slow, costly, hard to share with DOE
Field Validation – reduce errors
Fields support processing
Screening and prioritizing
Interface to DOE/HSS
Includes picture objects
Submit Button linked to LL Archive and Process
Becoming standard at facility level

24. More Focused Equals More Effective LANL Mirror (Quarterly Publication)
Translating Events into Actionable Information
Events binned by systems and processes
Patterns and trends (incidents, occurrences, causes)
Year-End and Two-Year Running Reflection
Use statistics to identify changes/trends
High-leverage lessons/actions aligned with greatest performance gaps
Recognition for developing and sharing lessons learned

25. More Focused Equals More Effective LANL Mirror: Events Causes Checklist for Improvements

26. Build and Use Indicators of LL Effectiveness Fact-based Improvement is the Objective
Use CAS Dashboard
LL Indicators
Processing time performance
Urgent within 24 hours
Caution within 7 days
Document Access Rate
Search Rate
Knowledge Inventory
Internal LL Generation Rate
LL Program Execution
Routine Process Milestones
Progress on Improvement Plan
Corrective Actions

27. Build and Use Indicators of LL Effectiveness Need measures than can help understand Cause and Effect

28. Build and Use Indicators of LL Effectiveness Need measures than can help measure use for Work Planning

29. Encourage Innovation LANL Performance Communication Center
Modernizing Communications
Chat with an Expert
Builds Knowledge Base
Blog
Role-based information
Relevant Tools
Relevant Requirements
Best Practices (coming soon)
Custom version of LL Form
Recognition

30. Encourage Innovation Sub-Unit Approaches Evolve as their Learning Processes Mature
R&D key lessons included in “On-Boarding” materials for new hires (might become a best practice)
Local LL Archives Linked to Institutional Archive
Project Management
Engineering
D&D
Maintenance
Facilities
Learning Teams (linked to HPI implementation)

31. Learning About Learning Progress in Lessons Learned System at LANL
Questions?