1. Considerations for Implementing Maintenance Line Operations Safety Assessments (M-LOSA)
Welcome to LOSA Management training. This training will provide an overview of Maintenance LOSA including basic characteristics that define LOSA, its intent, and how it is implemented.
August 2014 (v6)

2. Agenda Who is involved? What is LOSA? Why LOSA?
Status Report and LOSA Products
Steps to Implement LOSA
Questions/Discussion?
This project is financially sponsored by the Federal Aviation Administration (FAA), under the Flight Deck Program Directive/Level of Effort Agreement with the Human Factors Research and Engineering Group (ANG-C1), formerly (AJP-61).

3. What is LOSA? 3

4. SMS LOSA is…. FOQA ASAP ASRS CASS LOSA
Line Operations Safety Assessment
FOQA
ASAP
SMS
A Line Operations Safety Assessment (LOSA) is: a tool for collecting safety data during normal airline operations. It is a means for a company to complete a self-assessment. Monitoring routine operations, the cornerstone of the LOSA process, addresses an important aspect of safety auditing, namely, that risks and human error can never be completely eliminated. Also allows for positive behaviors to be reinforced.
LOSA is based off of the Threat and Error Management Model
Normal Working Environment’
Capture both Bad and Good
LOSA complements other safety programs:
ASAP (Aviation Safety Action Program)
ASRS (Aviation Safety Reporting System)
FOQA (Flight Operations Quality Assurance)
CASS (Continuing Analysis and Surveillance System) etc…
Under an SMS program, Safety is identified formally as a core business value. In a business process, you Plan, Implement, Measure, and Evaluate and then Re-plan.
Plan
Implement through the use of training inside the concepts of Threat and Error Management (TEM).
Measure through the use of LOSA in terms of safe versus at risk behaviors.
Evaluate the LOSA results (and all of the other data you collect).
Re-plan - Determine where you have risks in your operation and develop a plan to reduce those risks.
ASRS
CASS
LOSA

5. How does LOSA relate to SMS?
Focus is on Safety Risk Management (observing the system during normal operations)
It is a predictive hazard identification system
Foundation is based on Threat and Error Management (TEM)
Supports the continuous improvement philosophy

6. Reactive Hazard Identification
Investigation of accidents and incidents.
Boeing Maintenance Error Decision Aid (MEDA) for line maintenance and heavy maintenance.
Boeing MEDA Workshops for back shop maintenance.
Boeing Ramp Error Decision Aid (REDA)
Responds to events that have already happened – Past
Investigation of accidents and incidents (Based on the belief that we should learn from our mistakes)
Most appropriate for:
Events caused by errors and violations
Situations involving failures in technology
Unusual events
These investigations must go beyond the triggering cause(s) to include contributing factors and findings as to hazards and risks.
MEDA
MEDA Workshops
REDA

7. Proactive Hazard Identification
Mandatory and voluntary reporting systems, safety assessment, and surveys
Quality Assurance Audits
Internal hazard reporting systems
Safety surveys
Actively seeks the identification of hazardous conditions through the analysis of the organization’s processes - Present
Mandatory and voluntary reporting systems, safety assessment, and surveys.
-Based upon the belief that system failures can be minimized by:
Identifying safety risks within the system before it fails.
Taking the necessary actions to reduce such safety risks.
Quality Assurance Audits
-Existing QA Assessment
-QA Assessment added to Assessment components of the SMS
Internal hazard reporting systems
Safety surveys

8. Predictive Hazard Identification
Confidential reporting systems, flight data analysis, and normal operations monitoring.
Digital Flight Data Recorder (DFDR) Analysis
Line Operations Safety Assessment (LOSA)
Flight
Maintenance
Ramp
Analyzes system processes and environment to identify potential future problems - Future
Confidential reporting systems, flight data analysis, and normal operations monitoring.
- Based upon the belief that safety management is best accomplished by looking for trouble, not waiting for it.
- Aggressively seek information from a variety of sources, which may predict emerging safety risks.
Digital Flight Data Recorder (DFDR) Analysis
Line Operations Safety Assessment (LOSA)
- Flight
- Maintenance
- Ramp 8

9. Intent of LOSA Observations
Observe day-to-day work behaviors during normal operations
Discover procedural and systemic threats and errors
Reduce injuries & equipment/aircraft damage
Generate baseline data, implement new changes, and generate follow-up data to assess effectiveness
Acquire data about actual day-to-day safety behaviors in real-time, normal operations.
Discover procedural/systemic flaws that might lower safety margins.
Reduce errors from occurring that lead to injuries, equipment/aircraft damage, airline process failures, and other events.
Provide baseline and follow-up frequency data to assess effects of safety programs.

10. Trusted & Trained Observers Targeted Enhancements
Management &
Labor Agreement
Trusted & Trained Observers
Peer-to-Peer
Observations
Inform
Workforce
LOSA
Characteristics
Non-punitive
Peer-to-peer observations
Audits conducted by external agencies and internal safety departments may result in altered and rehearsed work behaviors which potentially leads to inaccurate data.
Peer-to-peer observation assessments encourage companies to capture more accurate data that reflects normal work behaviors.
Systematic observation
The LOSA observation guidance, training, and forms provided at describe a systematic method, Threat and Error Management, for conducting LOSA observations.
Comments written by the observer should have sufficient detail to allow others to understand the task and all its activities.
The observers need to describe the environmental conditions and activities surrounding the technicians’ and ramp workers’ behavior so that performance can be understood in full context.
Secure and Private
In order to maintain confidentiality, the organization should have a trusted data repository.
This site can be in-house, such as that used for other confidential data, or it can be offsite.
The goal is that no individual observations will be misplaced or improperly disseminated through the organization.
Non-punitive
LOSA observers are required to not record names, aircraft tail numbers, or any other information that can identify a team or individual.
The purpose of LOSA is to collect safety data, not to punish technicians and ramp workers.
Trusted and Trained observers
Primarily, frontline employees e.g., line technicians, base maintenance technicians, ramp workers, conduct LOSA observations.
It is critical to select observers that are respected and trusted within the organization to ensure line acceptance of LOSA.
After observers are selected, they are trained and calibrated in the LOSA methodology.
Volunteers
All LOSA observations are collected with voluntary technician and ramp worker participation.
Before conducting LOSA observations, observers should first ask the technicians and ramp workers for permission to be observed.
If an organization conducting a LOSA has an unreasonably high number of declines, this should serve as an indicator that there are critical “trust” issues to be resolved.
Management & Labor Agreement
In order for LOSA to succeed as a viable safety project, there needs to be support not only from the management side, but also from the technicians and ramp workers.
The joint sponsorship provides a “check and balance” for the project to ensure that change, as necessary, will be made as a result of LOSA data.
Targeted Enhancements
The final product of a LOSA is the data-derived targets for enhancement based on emergent patterns in the data.
It is then up to the organization to develop an action plan based on these targets, using experts from within the organization to analyze the targets and implement appropriate change strategies.
Inform Workforce
In order to ensure long-term success of LOSA, organizations should communicate the results back to the frontline employees.
Employees will want to see not only the results of the observations, but also management’s plan for improvement.
Secure & Private
Systematic
Observations
Targeted Enhancements
Volunteers 10

11. Why LOSA? Safety Issues Cost Issues
There are 2 ways to sell this program:
1. Safety Issues – Increase safety
2. Cost Issues – Reduce costs

12. Maintenance-Related Accidents
Why LOSA – Safety Issues
Maintenance-Related Accidents
Maintenance system failures, due to errors and/or violations, can affect safety of flight:
1. Primary Cause of an accident. Accident is due to the maintenance/inspection failure. Accident is not in any way due to flight crew action.
2. Contributing Factor to an accident. Accident chain begins with a maintenance/inspection failure that is incorrectly handled by the flight crew, ultimately ending up as an accident (Primary Cause is pilot error). 12

13. Accident: Maintenance is Primary Cause
Why LOSA – Safety Issues
Accident: Maintenance is Primary Cause
Toronto, Canada crossing the Atlantic Ocean, 2001
Airbus A330
Engine fuel leak
The Airbus A330 departed from Toronto, Canada bound for Lisbon, Portugal.
The flight ran out of fuel over the Atlantic Ocean and glided to emergency landing in the Azores carrying 293 passengers and 13 crew members.
There were 0 fatalities; 18 injuries, most minor, none life threatening.
Eight of twelve tires were blown out.
The investigation concluded that the accident was caused by a fuel leak in the number two engine.
The leak was caused by an incorrect part installed in the hydraulics system by the maintenance staff.
The part, adapted from a similar engine, did not maintain adequate clearance between the hydraulic lines and the fuel line.
Over time, vibration in the hydraulic lines degraded the fuel line and caused the leak.

14. Accident: Maintenance is Primary Cause (cont’d)
Why LOSA – Safety Issues
Accident: Maintenance is Primary Cause (cont’d)
The Portuguese, French, and Canadian investigated the accident.
The investigation concluded that the accident was caused by a fuel leak in the number two engine.
The leak was caused by an incorrect part installed in the hydraulics system by the maintenance staff.
The part, adapted from a similar engine, did not maintain adequate clearance between the hydraulic lines and the fuel line.
Over time, vibration in the hydraulic lines degraded the fuel line and caused the leak.
Consequences:
Air Transat was fined CAN 250,000.
DGAC and FAA issued Airworthiness Directives which led to changes in flight manuals of several Airbus models stressing that crews check any fuel imbalances for fuel leaks before opening the cross-feed valve.
Immediate Actions:
Airbus modified its computer systems so that the on-board computer now checks all fuel levels against the flight plan. A warning is issued if more fuel is being lost than the engine can consume
Rolls-Royce issued a bullet advising of the incompatibility of the affected engine parts.
Right Engine Pipe Contact
Fuel Pipe Crack and Scratches
*Photo courtesy of Aviation Accidents Prevention and Investigations Department, Government of Portugal

15. Maintenance as a Primary Cause
Why LOSA – Safety Issues
Maintenance as a Primary Cause
Data provided in a recent Advance-Notice of Proposed Amendment No xx/2007 from EASA*.
Primary Cause of HF-Related Accidents # of Accidents % of Total
Design 135 3%
Production 101 2
Operations (flight crew related)
Maintenance 416 8
ATM 66 1
Dispatch 18 0
Loading 129 2
TOTAL HF-Related Accidents
Non-HF-Related Accidents
TOTAL
*Data obtained from EASA Safety Analysis & Research. It is based on airliner accidents only, covering the period to January Only reports where causal factors have been positively identified are included in the analysis.

16. Accident: Maintenance as a Contributing Factor
Why LOSA – Safety Issues
Accident: Maintenance as a Contributing Factor
Lima, Peru 1996
Tape left on static ports
This event involving a 757 is a good example of how it is often a chain of errors that leads to an event. In this case, the airplane was in the paint shop to be polished.
Tape had been used to cover the static port to prevent entry of foreign material. The AMT had not signed the log book indicating that the work was not complete.
Another airplane developed problems on the gate, necessitating a change of equipment. The individual seeking replacement equipment failed to check the log and assumed the airplane was ready to go when it was not.
The pilot did not see the tape during the walk around and failed to abort the takeoff even though the primary instruments were not working.
The pilot lost control over the pacific and the airplane crashed. While cause of the accident was officially determined to be “pilot error, loss of control”, the actions in the paint shop clearly played a significant role in the chain of events.

17. Why LOSA? Safety Issues Cost Issues
There are 2 ways to sell this program:
1. Safety Issues – Improve safety (and efficiency)
2. Cost Issues – Reduce costs 17

18. Safety Costs of Maintenance Events
Why LOSA – Cost Issues
Safety Costs of Maintenance Events
International Air Transport Association (IATA) Safety Reports

19. Costs of Maintenance Error
Why LOSA – Cost Issues
Costs of Maintenance Error
Maintenance error caused…
20% to 30% of in-flight shutdowns (IFSDs) at an average cost of US $700,000 per IFSD
50% of flight delays due to engine problems at an average cost of US $13,000 per hour
50% of flight cancellations due to engine problems at an average cost of US $99,000 per cancellation

20. Understanding Shortcuts Identify Design Problems
Identify and Manage Threats
Observers note events in the operational environment (e.g., adverse weather, incomplete documentation, time pressure, and how they are managed by technicians).
Assess Training Effectiveness
Data provided by training programs can provide insight on whether training concepts are learned, but not whether they are actually practiced.
A LOSA provides that operational information, which can be reviewed from a training perspective to understand which areas of training, if any, are not transferring successfully.
Check the Quality and Usability of Procedures
A LOSA provides insights about potential problems with procedures. For example, if 50 percent of observed technicians make the same error, then the evidence suggests a problem with the procedure.
Involves Employee Groups
As subject matter experts in their work environments, technicians and ramp workers are in the best position to identify and record threats and errors.
Complements Existing Safety and QA programs
LOSA is distinct from - but complementary to - other safety programs such as voluntary reporting systems (e.g., ASAP).
Other programs generally rely on outcomes to generate data. By contrast, LOSA samples all activities in normal operations.
Assess Safety Margins
Threats and errors that are mismanaged can result in undesired states, if sufficiently serious.
A maintenance error that is not corrected before dispatch is an example of an undesired state, also known as an accident and incident precursor.
A LOSA provides data about the prevalence and management of these incident and accident precursors.
Identify and Manage Errors
Observers note errors arising from within the organization’s own operations and how they are managed (e.g., operational time pressure, dispatch errors, aircraft malfunction/minimum equipment list (MEL) items, and problems with interruptions, other technicians, and the ramp).
Provide a Baseline for Organizational Change
LOSA results provide baseline and outcome measurement data against which organizational interventions can be measured.
A follow up LOSA provides a new set of results that will show whether the organizational changes were effective in reducing certain threats, errors, and/or undesired states.
Identify Design Problems in the Human/Machine Interface
A LOSA captures aircraft design, tooling design, and ground service equipment design issues for which technicians and ground servicing personnel have developed “workarounds.” Some of these interface issues may be able to be fixed so that no workarounds are required
Understanding Shortcuts
With experience comes expertise; frontline employees learn ways to save time and be more efficient. These techniques are rarely seen in a traditional audit, when performance is usually done by the book. A LOSA provides an opportunity for the organization to capture collective expertise from within the frontline employee group, and then share that information with all its employees through formal organization communication channels. Using LOSA, false expertise – the adoption of a shortcut or workaround that is flawed in its safety assumptions - can also be identified and remedied.

21. Status Report and Success Stories…
LOSA development was initially started in 1991 at the University of Texas at Austin (UT-Austin) with funding from the FAA. The development of LOSA stemmed from a request by Part 121 carrier to validate the operational impact of its three-day Crew Resource Management (CRM) training course. Analysts soon realized that existing data collection methods did not assemble adequate information regarding flight crew adherence to standard operating procedures (SOPs) and environmental influences on flight crew performance. To explore the effectiveness of CRM training transfer, a partnership was established and the goal was to develop a line audit methodology utilizing jump-seat observations on regularly scheduled flights (i.e., LOSA).
After several years of development and refinement, LOSA has evolved into a strategy to provide safety data comprised of normal operations in technical and human performance areas. The LOSA observations provide data to develop countermeasures to operational threats and errors (ICAO, 2002). It has since been used and validated by many international airlines and is now recognized as a key element in an airline’s Safety Management System (SMS). 21

22. Status Report: LOSA Progress
Support Materials are ready
Available on the FAA’s Human Factors in Aviation Maintenance website:
Active users of Mx LOSA
COAIR/United
Considering LOSA and had Initial Briefing
Alaska Airlines Garuda Indonesia
FedEx China Airlines
Delta Air Lines EVA Airways
UPS Qantas
Garuda Airlines Korean Air
Jet Blue Airlines Etihad Airways
American Airlines Gulf Air

23. Maintenance Success Stories
M-LOSA findings help to make deactivation procedures more workable, efficient, and safer.
Before: Leading edge device de/reactivation procedures took three hours to properly tag out without individual sign-offs.
After: With sign-offs, this modified process takes between thirty and forty-five minutes to complete.
This success story is from the company formerly known as Continental.
An M-LOSA Assessment identified this inefficiency, which was then addressed by Tech Publications by rewriting their deactivation/reactivation procedures.
The new procedures also help to standardize the process to avoid problems caused by shift changes (deactivation and reactivation are often carried out by different shifts) and interruptions. This deactivation/reactivation procedure has been implemented in the airline’s entire fleet.
As your company implements LOSA, capture your own data that can demonstrate a return on investment of this program. Document your own success stories. 23 23 23 23

24. LOSA Products Ready to use
Forms
LOSA Products
Database Software
Training
In addition to forms, database software, and training; an implementation guideline for M-LOSA and R-LOSA programs and marketing materials i.e., posters are available.
All products are available on the FAA’s Human Factors in Aviation Maintenance website:
Ready to use 24

25. Forms Available forms are: Mx LOSA Observation forms
Ramp LOSA Observation forms

26. LOSA Forms
Developed by reviewing and aggregating procedures from flight deck LOSA (or similar programs) and related M-LOSA & R-LOSA references.
Refined during extensive taskforce discussions & beta testing.
M-LOSA & R-LOSA references refers to forms used at then existing M-LOSA and R-LOSA programs or LOSA themed programs, interviews/focus groups with the subjective matter experts, and other LOSA publications. 26

27. Training Maintenance LOSA (M-LOSA) Ramp LOSA (R-LOSA) LOSA Awareness
Database & Reporting

28. LOSA Training LOSA Awareness LOSA Procedures Scenario-based Practice
Database Familiarization
The training provides an understanding of the theory behind TEM and LOSA and gives an example of a LOSA observation based on a real world scenario. The Assess mentor will practice performing LOSAs on several real world scenarios and then enter the observations into the software tool.
LOSA Procedures refer to a LOSA implementation flowchart (a visual guide), implementation guideline, written protocols for the observations and project management tools for the LOSA managers.
The training materials are available for free at: 28

30. Maintenance LOSA Database Software
Records information about the observation
No identifying information
Analysis and report generation
Data exportable to Excel and other statistical software
Should have local analyst for best results
No “data-sharing” necessary
1. Basic information about the organization (name, address, phones... etc), and contact person (s) for that organization (name, title, phones, ...etc)
2. A set of tables that would contain all definitions for all 'forms'.
3. A set of tables for all known threat codes.
4. A set of tables for all know error codes. 30

32. Steps to Implement LOSA32

33. This is a broad overview of the recommended Steps to implement a LOSA
Form an initial development team/LOSA Steering Committee
Begin Marketing LOSA – utilize LOSA posters & brochures
Seek Union Involvement
Learn from LOSA implementation at other companies, including conducting GAP analysis in own company
Use existing or create a secured database
Train all staff (including observers) on LOSA
Schedule observation dates
Select Trusted and Respected Observers
Carry out observations
Carry out data verification
Analyze data
Write and distribute reports for all listing both strengths and weaknesses
Develop enhanced policies, procedures, training, etc. based on the findings

34. Strategies to Insure Trust and Respect
Promote LOSA
Emphasize that LOSA is not for disciplinary purposes and that the forms do not include any personal information.
Select credible, respected observers.
Ask crew if it is OK to do the LOSA observation.
Communicate the LOSA results in a timely manner.
Use the information—create action plans, implement changes, and evaluate results.
Promote LOSA – Management must encourage and support it.
Emphasis on safety and not disciplinary purposes.
Select credible, respected observers.

35. Key Reminders Ensure Buy-in Fundamentals are critical
Communication to everyone is critical
Available Training
Consistency is essential
One false step with discipline is significant
Start small/target areas for improvement

36. Conclusion
The following products are available for implementing LOSA at airlines…
Marketing materials (posters & brochures)
Implementation Guideline
Observation forms
Database software
Training
LOSA is a predictive hazard identification system for your SMS that can:
Reduce your costs,
Improve safety, and
Improve efficiency
This project is financially sponsored by the Federal Aviation Administration (FAA), under the Flight Deck Program Directive/Level of Effort Agreement with the Human Factors Research and Engineering Group (ANG-C1), formerly (AJP-61).
In addition to available observation forms, database software, and training; a LOSA implementation flowchart (a visual guide), implementation guideline, written protocols for the observations and some project management tools for the LOSA managers are included in the guideline.

37. Questions/Comments? 37

38. This concludes the introduction to Line Operations Safety Assessment.
Visit the FAA’s Human Factors in Aviation Maintenance website:
Acknowledgements: ATA/FAA Maintenance and Ramp Human Factors Committee