Presentation on theme: "Jennie J. Gallimore, Ph.D. June 24, 2009 NASA Langley"— Presentation transcript:
1Jennie J. Gallimore, Ph.D. June 24, 2009 NASA Langley Head Position and Frame of Reference in Flight: The Opto-kinetic Cervical ReflexJennie J. Gallimore, Ph.D.June 24, 2009NASA Langley
2Topics Spatial Disorientation Attitude Indicator OKCR Research Considerations for Cockpit DisplaysOther On Going Research at WSU
3Spatial Disorientation The inability of the flight crew to correctly perceive attitude, altitude, or airspeed of the aircraft in relationship to the earth and other points of reference.SD has been categorized into three types.Type I unrecognized (where most mishaps are classified)Type II recognizedType III incapacitating
4Spatial Disorientation Accidents US Air Force , 20.2% of accidents, 60 lives, 1.4 billion dollars [Davenport00].US Army reported 27% [Kuipers90].US Navy and Marine Corp, 26%, 101 accidents [Johnson00]Three times as many lives were lost for SD related mishaps compared to non-SD related mishaps.Mishap data also indicate that pilots who experience SD are very experienced, and the mishap rate has not decreased in the last 20 years.
6First Mechanical Flight Instrument Sperry’s bank and turn indicator, 1918Worked best in clear weather
7First “Blind” Sortie First ‘Blind’ sortie, takeoff to landing Sep 24th 1929 under direction of Guggenheim in NY (Mitchel Field)First ‘Blind’ sortie, takeoff to landingFirst use of artificial horizon, Kollsman altimeter, and directional gyro
8Attitude Indicator Level flight 20-degree turn This is an example of the current attitude indicator in US aircraft. It is the display that is used to kept the pilot oriented with respect to the world. In this figure our perspective is from behind the aircraft. The figure on the left shows the plane(the yellow symbol) level with the horizon. The figure on the right shows the plane in a right bank. The aircraft symbol is still level, but the horizon is rolled to the left.The idea behind the original 1930’s design of this instrument was that the horizon on the display would provide the pilot with a pictorial representation of what the pilot would see if they were looking out of the cockpit at the horizon.Lets look at this more closely.Level flight20-degree turn
9Attitude Indicator Example Real WorldPlane movesHorizon remains staticIndicated WorldPlane remains staticHorizon movesIndicator reverses realityDangerous if visual referenceis lostPilot disorientationControl reversal errorsIn the real world the frame of reference pilots use to keep oriented includes a static horizon and and moving airplane.On the indicator, the plane (the yellow line in the center) remains fixed, and the moving element is the horizon. If you move the control stick to the right to put the plane in a right bank, the display moves to the left. Therefore, there is no control-display compatibility. But the idea was that when the aircraft is in a bank (such as in this right bank) the real horizon would appear tilted as you looked out. However, this idea was based on the untested assumption that your head remains fixed with the aircraft axis. We have recently learned through research conducted at WSU, that when the plane is banked, the pilot tilts his head in the opposite direction to keep the horizon level and fixed on his visual system. This head tilt is a reflex, that most pilots are unaware of, and is called the opto-kinetic cervical reflex. So in all US aircraft, pilots fly with an attitude indicator with no display control compatability, and without an accurate pictorial representation.When transitioning from looking outside at the stable horizon to the instrument (with a moving horizon, that is tilted) the pilot must instantly switch his cognitive frame of reference. This can lead to disorientation and control reversal errors. Control reversal errors are when a pilot pushes the stick in the wrong direction.In the diagram above, if the pilot lost sight of the horizon and pushed the stick to the right in order to level the display, he would roll the airplane over, and may be uaable to recover. Highly trained pilots with thousands of hours of flight time commit control reversal errors. Less experience general aviation pilots such as JFK, probably commit even more.
10To understand what may have happened to JFK, lets look at pilot spatial awareness models. The attitude indicator is the display you see at the bottom of the picture on the left. It is the display that is used to kept the pilot oriented with respect to the word. Is is composed of a pictorial representation of the ground and the sky. There is an aircraft symbol in the middle. This was developed back in the 1930’s under the idea that the attitude indicator provides the pilot with a pictorial replica of what he sees when looking out of the cockpit In this display, the aircraft is fixed and the horizion moves as the plane banks. If the pilot keeps his head aligned with the aircraft, then he will see the representation in the middle of the figure, which you notice matches the picture on the attitude indicator. Here is a video which illustrates this concept. It is a Navy hornet flying and banking. From this perspective the idea is that the plane stays fixed and the horizon moves. One problem that was noted with the design of this display early on is that the movement of the control is not compatible with the movement of the display. In other words when you move the stick to the right to put the plane in a right bank, the movement on the display is to the left. But it was felt that since it was pictorially accurate, that over rode the disadvantage of display control incompatability.Again this theory of pictorial reality is true if the pilot keeps his or her head aligned with the aircraft.Now we will look at a C130 pilot who is banking his plane to the left. Notice how he tilts his head with the horizon. Well that is fine, but someone pulling Gs would certainly not be able to tilt their head! Lets watch a blue angel!
12What the pilot sees when he looks out of the cockpit is a fixed horizon, and his plane is moving. With this perspective, there is control/display compatibility. The cockpit structures move in the direction of the bank, and the horizon is stable.So what does this mean?But notice, the view is opposite that of the attitude indicator. So what we really have in all aircraft in the US is an attitude indicator that does not represent pictorial reality and it also has no control display compatibility.
13Opto-Kinetic Cervical Reflex (OKCR) Pilots align their heads toward the horizon during Visual Meteorological Conditions (VMC) flight.Pilots do not tilt their heads during Instrument Meteorological Conditions (IMC) flight.Visual to Instrument transition can cause reversal errors.The tilting of the head to keep the horizon stable, was termed the OKCR, and the was investigated for the first time at WSU back in 1994.Because of this reflex:Pilots align their heads toward the horizon during Visual Meteorological Conditions (VMC) flight.Pilots do not move their heads during Instrument Meteorological Conditions (IMC) flight.Therefore:Visual to Instrument transition can cause reversal errors.
14Head TiltPatterson (1989) noticed that pilots align their heads with the horizon.If they are aligning their heads with the aircraft then the view from the windscreen is a fixed horizon (not moving).
15Opto-Kinetic Cervical Reflex (In-flight) Horizon Linewith 73 degreesof bank angleF/A-18 aircraft (Blue Angel)73 degrees of bank (VMC, +Gz Turn).OKCR Head tilt = 31degrees away from the Gz axis.
16WSU Research Investigating Head Tilt Patterson (1995, 1997)Smith et al (1997)Merryman et al (1997)Gallimore et al (1999, 2000)Liggett & Gallimore (2001)Gallimore, Liggett & Patterson (2001)Others since
17OKCR Studies Author Plat-form Visual Field Size Instru-ments VMC Task OCKR Found?IMCOKCRFound?UACRE%SubsPatterson(1995)Fixed aircraftsimFull dome180oHDDAIXYesNo65%16Smith et al. (1997)Merryman et al. (1997)F-15aircraftRealworldHUD9Braithwaiteet al (1998)MovingHeli-copter SimHalf dome160o H FOVNVG25%20Gallimore et al. (1999)31%12Gallimoreet al. (2000)AII26
18Horizon Roll Vs. Head Roll for Low-Level Route Patterson et al.
19Three graduate studies: Patterson, Merryman, Smith
24Reversal ErrorTendency for pilots to mistake motion of the artificial horizon as a relative motion of the wings.Pilots roll or pitch the aircraft in opposite direction.Researchers who have documented this errorFitts and Jones (1947)Johnson and Roscoe (1972)Roscoe and Williges (1975)Roscoe (1986) - Boeing 747 accident
25Sensory-Spatial Conflict and Control Reversal Error (Patterson et al findings) Experienced U.S. military rated pilots commit 25-65% reversal errors.Likelihood of reversal errors by general aviation pilots is probably even greater.A reversal error can lead to flight into terrain or a graveyard spiral.This is likely what happened to the pilot of Air India and to John F. Kennedy, Jr.References on reversal errors:Patterson, et al, 1997Braithwaite,et al, 1998Gallimore, et al., (1999)Liggett & Gallimore (in press)Control reversal error during IMC “out” to “in” visual transition.During transition, when the cognitive switch between those spatial strategies switches, pilots make reversal errors. That is, they put in a stick movement opposite to that which they need to level the plane. Here is an example of a military pilot performing a control reversal error in a simulator. In this exercise, the pilot was flying behind a lead aircraft. The lead aircraft was moving into an unusual attitude, and we took away the lead aircraft. The pilot must look at their instruments and return to level flight as quickly as possible when they lose the lead. You will hear the experimenter say “ok take it away” which means the lead is removed. What you see is the attitude indicator go in one direction, reverse, then reverse again and the pilot does a roll to recover.This was not the fastest way to return to wings level.
26Number and Magnitude of Reversal Errors Gallimore, et al findings 40 Degrees60 Degrees100 Degrees9 errors out of 248 errors out of 246 errors out of 24VMC4(17.39%)1(.04%)5(20.83%)4(17.39%)5(20.83%)4(17.39%)IMCAverage reversalerror magnitude9.34 oCombinedAverage reversalerror magnitude28.96 oAverage reversalerror magnitude9.30 o
27TransitionsWhat happens during the transition from visual to instruments?The pilot’s view of the cockpit suddenly becomes stationary as his view of the display’s artificial horizon begins moving.Pilots must instantly reverse their orientation strategy.Pilots sensory-spatial compatibility between the control stick motion and visual feed back.
28SummaryPilots reflexively tilt heads toward horizon during VMC roll maneuvers.Head movement acts to stabilize retinal image.Generated by motion on retina, not vestibular.Stabilized horizon is the primary visual cue.Peripherally viewed cockpit structures secondary cues.Secondary cues move with airframe.Control movement compatible with secondary cues.
29Summary (Cont.)Beyond 40 degrees of aircraft roll there is a decrease in head displacement, so pilots can not stabilize the horizon.Horizon acceleration, stabilization of secondary cues.Sudden switch may lead to false perceptions.When transitioning from visual to instrumentsmotion reversal b/w outside and inside visual cuescontrol display incompatibilityneed to switch cognitive model
30How does OKCR affect current display technologies? Head down Attitude IndicatorReversal errorsHUDHead may tilt out of the HUD eye box and pilot may not see a pull up X.
31HUDThe Head Up Display (HUD) presents symbols to the pilot, displaying them over the real world.
34How does OKCR affect current display technologies? (cont) NVGHUD symbology on the NVG. Head movements are not tracked. As pilot changes head position, display horizon line is no longer conformal to the real horizon.Pilots see HUD information designed for fixed on-axis aircraft viewing regardless of head position. Pilots may not realize they are not flying in the direction they are looking.
35Research IssuesWhat frames of reference are important for a pilot to maintain orientation?World - world is fixed and everything moves within it.Aircraft - aircraft is fixed and everything moves around it.Pilot - pilot is fixed and everything moves in relation to him.One of the most fundamental issues of maintaining spatial orientation (SO) is to identify the most desirable frame of reference for establishing a correct attitude in a pilot’s internal model of SO. There have been numerous studies to characterize a visual reflex called the optokinetic cervical reflex (OKCR) that implies that pilots use a world frame of reference to orient themselves when looking at real world visual cues. When pilots use instruments to determine orientation, the information is portrayed in an aircraft frame of reference. This change in frames of reference when pilots transition from visual meteorological conditions (real-world cues) to instrument meteorological conditions (instrument cues) may be directly affecting their SO model, leading to spatial disorientation (SD)
36Research Issues What symbology is appropriate for HMDs? HUD symbology is being considered for use on HMDs.HUD symbology is being used on NVGs.How do sensory reflexes affect perceived frame of reference?OKCR, under VMC pilots align their head with the horizon.
37Research IssuesHow do visual frames of reference interact with vestibular and proprioceptive inputs to provide the pilot with an "awareness" of their orientation?What contributing cognitive factors affect spatial orientation?How will HMD attitude symbology affect frames of reference in VMC and IMC?How will transitions be impacted?How can we detect when a pilot is spatially disoriented?
38Research Issues for HMD Symbology Design What spatial sensory reflexes and visual illusions influence pilot’s perception of frame of reference?Will cognitive capture affect pilots perceptions of frame of reference? Will cognitive capture result in more transitions between symbology and the real world?When pilots transition between a perceived stationary horizon (real world cues) to a moving symbol horizon on the HMD, do they perceive the horizon symbol as stationary?What type of symbology will help provide the perception of a stationary horizon?
39Research Issues for HMD Symbology Design If HMD symbology is used for attitude information as well as targeting, how will switching between these tasks affect frame of reference?Will pilots have a greater risk of spatial disorientation if they look off-axis more often?How will secondary flight cues be affected by use of HMDs?What current or new measures should be employed to determine if a pilot is spatially disoriented?
40HMD Research and the OKCR Experiment ITest adequacy of Mil-Std HUD symbology presented on a see-through HMD during various tasks.VMC flight taskPilots were instructed to bank at specific angles, rather than to bank around a waypoint.12 SubjectsHMD Kaiser Pro – Binocular HMD, 40o circular FOV, 100% overlap, 1280 x 1024 resolution.
41HMD Research and the OKCR Experiment IIInvestigate visual cues in an immersed HMD simulation system using HUD symbology.VMC Flight taskVaried resolution (640 x 480 & 800 x 600), HUD symbol size (small and large)Pilots instructed to follow a yellow track line over Pensacola, FL6 subjectsVirtual Research V8 HMD system, 48o H x 32o V, 100% overlap
42HMD Research and the OKCR Experiment IIIInvestigate the effects of non-congruent motion on performance in an immersive HMD system.VMC flight task flown on land and on Navy mind sweeper in Pensacola Bay.Pilots instructed to follow a yellow track line over Pensacola, FL9 subjectsSony i-glasses , 24o H x 18o V, 100% overlap, 789 x 230 Resolution
43HMD ResultsEXP IAnalysis of the data and pilot comments indicated that subjects did not tilt their head because they were using the bank scale symbology on the HMD to determine and hold their bank angle prescribed during the task via a verbal command to bank their aircraft to a certain bank angle and to maintain that bank until instructed to level out. Pilots did not have to rely on any outside visual cues to conduct the task resulting in an instrument-oriented task. In all previous studies pilots were flying tasks that required them to maintain awareness of the real-world visual scene in order to bank the aircraft.EXP IIThere was a significant difference in head tilt as a function of aircraft bank, indicating an OKCR response under VMC as illustrated in Figure 3. While the OKCR response is obvious, the magnitude of the response is much smaller than previous studies using low level flight tasks. The OKCR response is very similar to results found by Gallimore et al. (2000)5 (Figure 2) in which pilots flew at higher altitudes.EXP IIIAs indicated in the figure, both land and shipboard HMD simulations produced significant OKCR responses. The OKCR differences between ship and land conditions were not significant. The results are very similar to results from Experiment II. Preliminarily results indicate that individuals who reported higher scores on their motion sickness surveys exhibited less OKCR during the shipboard simulation compared to the land based condition.
44OKCR DifferencesDifferent visual scenes/cues cause difference in pilot OKCR responseReducing FOVManipulating altitudeAmount of head tilt depends on amount of retinal movement.Reduction in peripheral vision may play a roleReducing FOV may reduce how compelling the visual horizon appears
45OKCR DifferencesImmersive HMD simulation studies did not provide any secondary visual cues (cockpit structures).Do pilots reduce head movements when they lack a stabilizing cue?If experiencing simulator sickness may reduce head movements.
46Control Reversal Errors HMD Liggett and Gallimore findings Overall CRE rate 28%, similar to previous studies.Magnitude range: 6 degrees to 201 degreesA conformal horizon symbol did not reduce CREs.Because we know they were not tilting in IMC, they still had to change frames of reference from world to aircraft.
47Control Reversal Errors HMD Liggett and Gallimore findings Dependent measure: Altitude ChangeSignificant differenceCRE group average: ft MSLNo CRE group average: ft MSLPilots with CREs obviously confused.Focusing on pitch and bank information in central part of symbology.Fail to scan airspeed and altitude information.
481. Recognition of pilot spatial awareness strategies 3. Avoidance and recognitionof Visual Illusions (perspective illusion)2. Avoidance and recognitionof spatial disorientation(VMC-IMC form/ reversal error)4. Design of flightdeck displaysAvoidance and recognition of visual illusions.
49Spatial disorientation factor Perspective (moon) illusion For the AVIANO mishap, I don’t have time to go over all the issues, but I would like to show you one factor that affected the pilot’s spatial awareness. This information comes from expert witnesses at the trial.This is an example of the Perspective (or Moon) illusion. Some of you may have experienced it before. When the moon is low and just over the horizon it appears to be very large. But if you took a picture, it would be the “normal” size.In this illusion, the box in the back looks larger than the one in the front, yet they are actually the same size.Things on the horizon are thrown out of perspective. Even it you know it is an illusion, you can’t turn it off. It can’t be trained away. It is very compelling.
50Example: Perspective Illusion This is a tape of the valley in AVIANO where the mishap took place. This tape was recorded by flying a helicopter over the valley at the same altitude that the plane flew. The prosecution recorded this tape. The cables were gone, and they put up two white weather balloons to show where he cables were located.Some basic information.The pilot was practicing a low level flight. He must keep his view outside the aircraft. It is mountainous terrain.It only takes 30 seconds to get from one end of the valley to the next.The mishap happens about 20 seconds into the valley.This tape is from a point in the valley when there is only 12 seconds before the pilot (flying 500 knots 900 feet per sec) reached the mishap site. The helicopter is flying at 400 knots.
51This is the same tape. But on it is a line which indicates the horizontal reference that the pilot would be fixating on or looking at in order to fly through the valley.The balloons are highlighted in red so you can see them.What do you see?As you start out it appears that the balloons are below the horizontal reference point, but as we proceed the balloons appear to rise. Are you below the balloons?You experienced the perspective illusion. The red balloons are not rising. The horizontal reference point is changing. Which you can see if I put my cursor at the place were we see the red ball, then when it is finished they are still level with my cursor.At all times the plane is actually 100 feet above the balloons. We never go below them. But to the pilot it appears as if you are going to hit the balloons. He sees the first cable, thinks he is going to hit it. He has a few seconds to respond, he pushes the nose down to go under it because the aircraft can’t go above in that time period, he missing the first cable, but hits the second on that is lower than the first.
52References Aviation Research Jenkins, J. C., and Gallimore, J.J. (2008). Configural display design features to promote pilot situation awareness in helmet-mounted displays. Aviation, Space and Environmental Medicine, 79,Stephens, M., Gallimore, J., and Albery, W. (2002) Spectral Analysis of Electroencephalographic Response to Spatial Disorientation. Proceedings of the 12th International Symposium on Aviation Psychology: Dayton OH. (pp ).Liggett, K.K. and Gallimore, J.J. (2002). The effects of frame of reference and HMD symbology on control reversal errors. Aviation, Space, and Environmental Medicine;73:Gallimore, J.J., Liggett, K.K. and Patterson, F.R. (2001). The Opto-Kinetic Cervical Reflex in Flight Simulation. Proceedings of the American Institute of Aeronautics and Astronautics Modeling and Simulation Conference and Exhibit, Aug 6-9, 2001, Montreal, Canada, Paper No: : pp 1-7. * Best Paper.
53References Aviation Research Liggett, K. and Gallimore, J.J. (2001) The OKCR and Pilot Performance During Transitions Between Meteorological Conditions Using HMD Attitude Symbology. In Proceedings of the Human Factors and Ergonomics Society 45th Annual Meeting, (pp ) Santa Monica. CA HFES.Gallimore, J.J., Patterson, F.R., Brannon, N.G., and Nalepka, J.P. (2000). The opto-kinetic cervical reflex during formation flight. Aviation, Space and Environmental Medicine 2000;71:Gallimore J. J., Brannon, N. G., Patterson, F.R., and Nalepka, J.P. (1999). Effects of FOV and aircraft bank on pilot head movement and reversal errors during simulated flight. Aviation, Space and Environmental Medicine, 70(12):Gallimore, J.J., Brannon, N.G., and Patterson F.R. (1998). The Effects of Field-of-View on Pilot Head Movement During Low Level Flight. In Proceedings of the Human Factors and Ergonomics Society 42nd Annual Meeting, Chicago, IL (pp. 6-10). Patterson F. R., Cacioppo, A.J., Gallimore, J.J., Hinman, G.E., and Nalepka, J.P. (1997). Aviation spatial orientation in relationship to head position and attitude interpretation. Aviation, Space and Environmental Medicine, 68(6),
54Other ResearchA Predictive Model Of Cognitive Performance Under Acceleration StressSubmitted to Aviation, Space, Environmental Medicine, June 09Three-Dimensional Technology for Space Operation ApplicationsMulti-modal Displays for Portraying Meta-Info to Support Net-Centric C2Process Control DisplaysVirtual PatientsCollaborative Computer Agents with Personality
55Acknowledgements CDR Frederick Patterson, Ph.D., Retired Naval Aerospace Medical Research LaboratoryUnited States NavyI would like to acknowledge Cmd Fred Patterson of the Naval Aeromedical Research Laboratory, who has funded some of this research and has provided some materials for this presentation.