1. Air Carrier Continuing Analysis and Surveillance System (CASS)
Validate Effectiveness of Maintenance Program

2. Objectives
Determine how to assess the effectiveness of the maintenance program
Identify effectiveness processes within the air carrier’s maintenance program
Describe the carrier’s procedures in its CASS for validating effectiveness of its maintenance program

3. Overview Surveillance Analysis Select data sets Collect data Purpose
Schedule

4. Primary Goals of a Maintenance Program
All maintenance performed by the air carrier or by other persons is performed according to the air carrier manual.
Competent personnel and adequate facilities and equipment are provided for the proper performance of maintenance.
Each airplane released to service is airworthy and has been properly maintained for operations in air transportation.

5. Verify Effectiveness of the Maintenance Program

6. Data Sets Selection What operational data to collect
Who will collect it
How to collect it
When to collect it
What to do with it (after it is collected)

7. Operational Data Definition
Generated as the result of flight operations
Helps determine if the maintenance program is producing the desired results

8. Collecting Data Assessing effectiveness of MP:
Collection of operational data
Measures the output of inspection and maintenance programs

9. Data Collection Process
Data Collection Systems
Collect
Store
Manage
Retrieve

10. Collecting Operational Data
What operational data to collect
Defined by carrier
Meets CASS requirements
Can be divided into Routine or Nonroutine

11. Collecting Operational Data
Who will Collect
How to Collect
Defined procedures
Quality data
Periodic audits
Documented
When to Collect
Frequency of collection
Periodic review of usefulness of data

12. Red Rock Airline Data Sources

13. Common Misconceptions
No reliability program, no requirement to collect data.
Manufacturers collected data, so “I don’t need to.”
If an organization has a reliability program and an audit program, it automatically has a CASS.
Integration between reliability program and CASS is not necessary.
“I’m a small operator with one aircraft that flies 10 hours per month. I know what’s going on.  I don’t need CASS.”

14. Analysis

15. Purpose of Analysis
Understand potential significance and relationship of all data sets
Identify causal factors
Analysis should include:
Consideration of human factors
May include expert input:
Technical
Engineering
Reliability

16. Analysis Schedule
The air carrier needs to clearly define the frequency at which data analysis will occur based on the type of operational data.

18. Red Rock Airline Data Analysis

19. Focus of Analysis Hazard identification Adverse indicators
Risk analysis
Risk assessment

20. Inspector Oversight
Review policies, procedures, instructions, and information
Determine desired outcome
Review data

21. Other Things to Consider
Analysis of operational data within alerting systems or warning systems when exceeding certain parameters.
Do not rely completely on these automated systems in favor of good judgment on the part of the analyst.
The FAA’s expectation of a CASS in this regard is that the air carrier has a complete, written procedure to review and analyze the operational data collected and to determine when corrective action is necessary.
Analysis of the operational data will produce findings. Now a decision must be made as to whether or not, or how, to address each finding.
Analysis of operational data within alerting/warning systems
Do not rely completely on these automated systems (use good judgment)
The FAA’s expectation of a CASS: Complete, written procedure to review and analyze the operational data and to determine when corrective action is necessary
Analysis produces findings
Make a decision as to whether or not, or how, to address each finding

22. Scenario As previously discussed:
Red Rock Airline is a large U.S. Part 121 domestic/flag air carrier
Outsources 70 percent of its aircraft maintenance
Aircraft experienced two “EFIS comparator monitor” caution messages
The crew followed standard procedures

23. Red Rock Airline “Incident”
14 May 2009
“EFIS comparator monitor” caution message
Aircraft Operating Manual procedure was followed by the crew
Alternate compass swing was accomplished
Found within limits
Aircraft was scheduled for a full compass swing
Within 700 flight hours, or
Before two “A” checks were completed

24. Red Rock Airline “Action”
20 May 2009
Aircraft was scheduled for a full compass swing to comply with the requirements
During the inspection:
Standby Magnetic Compass and First Officer’s Flux Valve were out of tolerance
Both parts were removed and replaced
Operational checks were accomplished

25. Red Rock Airline “2nd Incident”
3 June 2009
While in-flight to Denver, the same aircraft experienced an “EFIS comparator monitor” caution message
Crew followed the Aircraft Operating Manual procedure

26. Red Rock Airline “2nd Action”
3 June 2009
Aircraft was ferried to Colorado Springs, Colorado
System was troubleshot and magnetized areas were found on the left wing near the flux valve
The following was accomplished, and the aircraft was returned to service:
Eleven (11) magnetized screws were removed
The area in question was demagnetized
The #1 Remote Compensating Unit was removed and replaced
A full compass swing was accomplished
The most recent audit of this contract maintenance provider indicated no findings for accomplishment of maintenance. The audit schedule shows the next audit is on a one-year cycle.

27. Red Rock Airline Operational Data 06/2009

28. Practice Exercise

29. Practice Exercise

30. Summary Surveillance Analysis Select data sets Collect data Purpose
Schedule

31. Objectives
Determine how to assess the effectiveness of the maintenance program (MP) within CASS
Identify the different types of data driving the effectiveness portion of CASS
Identify effectiveness processes within the air carrier’s maintenance program

32. QUESTIONS?