Presentation is loading. Please wait.

Presentation is loading. Please wait.

Jennie J. Gallimore, Ph.D. June 24, 2009 NASA Langley

Similar presentations

Presentation on theme: "Jennie J. Gallimore, Ph.D. June 24, 2009 NASA Langley"— Presentation transcript:

1 Jennie J. Gallimore, Ph.D. June 24, 2009 NASA Langley
Head Position and Frame of Reference in Flight: The Opto-kinetic Cervical Reflex Jennie J. Gallimore, Ph.D. June 24, 2009 NASA Langley

2 Topics Spatial Disorientation Attitude Indicator OKCR Research
Considerations for Cockpit Displays Other On Going Research at WSU

3 Spatial Disorientation
The inability of the flight crew to correctly perceive attitude, altitude, or airspeed of the aircraft in relationship to the earth and other points of reference. SD has been categorized into three types. Type I unrecognized (where most mishaps are classified) Type II recognized Type III incapacitating

4 Spatial Disorientation Accidents
US Air Force , 20.2% of accidents, 60 lives, 1.4 billion dollars [Davenport00]. US Army reported 27% [Kuipers90]. US Navy and Marine Corp, 26%, 101 accidents [Johnson00] Three times as many lives were lost for SD related mishaps compared to non-SD related mishaps. Mishap data also indicate that pilots who experience SD are very experienced, and the mishap rate has not decreased in the last 20 years.

5 First Flight Instrument?
Wright 1909 Military Flyer Slip Ribbon (first flight instrument)

6 First Mechanical Flight Instrument
Sperry’s bank and turn indicator, 1918 Worked best in clear weather

7 First “Blind” Sortie First ‘Blind’ sortie, takeoff to landing
Sep 24th 1929 under direction of Guggenheim in NY (Mitchel Field) First ‘Blind’ sortie, takeoff to landing First use of artificial horizon, Kollsman altimeter, and directional gyro

8 Attitude Indicator Level flight 20-degree turn
This is an example of the current attitude indicator in US aircraft. It is the display that is used to kept the pilot oriented with respect to the world. In this figure our perspective is from behind the aircraft. The figure on the left shows the plane(the yellow symbol) level with the horizon. The figure on the right shows the plane in a right bank. The aircraft symbol is still level, but the horizon is rolled to the left. The idea behind the original 1930’s design of this instrument was that the horizon on the display would provide the pilot with a pictorial representation of what the pilot would see if they were looking out of the cockpit at the horizon. Lets look at this more closely. Level flight 20-degree turn

9 Attitude Indicator Example
Real World Plane moves Horizon remains static Indicated World Plane remains static Horizon moves Indicator reverses reality Dangerous if visual reference is lost Pilot disorientation Control reversal errors In the real world the frame of reference pilots use to keep oriented includes a static horizon and and moving airplane. On the indicator, the plane (the yellow line in the center) remains fixed, and the moving element is the horizon. If you move the control stick to the right to put the plane in a right bank, the display moves to the left. Therefore, there is no control-display compatibility. But the idea was that when the aircraft is in a bank (such as in this right bank) the real horizon would appear tilted as you looked out. However, this idea was based on the untested assumption that your head remains fixed with the aircraft axis. We have recently learned through research conducted at WSU, that when the plane is banked, the pilot tilts his head in the opposite direction to keep the horizon level and fixed on his visual system. This head tilt is a reflex, that most pilots are unaware of, and is called the opto-kinetic cervical reflex. So in all US aircraft, pilots fly with an attitude indicator with no display control compatability, and without an accurate pictorial representation. When transitioning from looking outside at the stable horizon to the instrument (with a moving horizon, that is tilted) the pilot must instantly switch his cognitive frame of reference. This can lead to disorientation and control reversal errors. Control reversal errors are when a pilot pushes the stick in the wrong direction. In the diagram above, if the pilot lost sight of the horizon and pushed the stick to the right in order to level the display, he would roll the airplane over, and may be uaable to recover. Highly trained pilots with thousands of hours of flight time commit control reversal errors. Less experience general aviation pilots such as JFK, probably commit even more.

10 To understand what may have happened to JFK, lets look at pilot spatial awareness models. The attitude indicator is the display you see at the bottom of the picture on the left. It is the display that is used to kept the pilot oriented with respect to the word. Is is composed of a pictorial representation of the ground and the sky. There is an aircraft symbol in the middle. This was developed back in the 1930’s under the idea that the attitude indicator provides the pilot with a pictorial replica of what he sees when looking out of the cockpit In this display, the aircraft is fixed and the horizion moves as the plane banks. If the pilot keeps his head aligned with the aircraft, then he will see the representation in the middle of the figure, which you notice matches the picture on the attitude indicator. Here is a video which illustrates this concept. It is a Navy hornet flying and banking. From this perspective the idea is that the plane stays fixed and the horizon moves. One problem that was noted with the design of this display early on is that the movement of the control is not compatible with the movement of the display. In other words when you move the stick to the right to put the plane in a right bank, the movement on the display is to the left. But it was felt that since it was pictorially accurate, that over rode the disadvantage of display control incompatability. Again this theory of pictorial reality is true if the pilot keeps his or her head aligned with the aircraft. Now we will look at a C130 pilot who is banking his plane to the left. Notice how he tilts his head with the horizon. Well that is fine, but someone pulling Gs would certainly not be able to tilt their head! Lets watch a blue angel!


12 What the pilot sees when he looks out of the cockpit is a fixed horizon, and his plane is moving. With this perspective, there is control/display compatibility. The cockpit structures move in the direction of the bank, and the horizon is stable. So what does this mean? But notice, the view is opposite that of the attitude indicator. So what we really have in all aircraft in the US is an attitude indicator that does not represent pictorial reality and it also has no control display compatibility.

13 Opto-Kinetic Cervical Reflex (OKCR)
Pilots align their heads toward the horizon during Visual Meteorological Conditions (VMC) flight. Pilots do not tilt their heads during Instrument Meteorological Conditions (IMC) flight. Visual to Instrument transition can cause reversal errors. The tilting of the head to keep the horizon stable, was termed the OKCR, and the was investigated for the first time at WSU back in 1994. Because of this reflex: Pilots align their heads toward the horizon during Visual Meteorological Conditions (VMC) flight. Pilots do not move their heads during Instrument Meteorological Conditions (IMC) flight. Therefore: Visual to Instrument transition can cause reversal errors.

14 Head Tilt Patterson (1989) noticed that pilots align their heads with the horizon. If they are aligning their heads with the aircraft then the view from the windscreen is a fixed horizon (not moving).

15 Opto-Kinetic Cervical Reflex (In-flight)
Horizon Line with 73 degrees of bank angle F/A-18 aircraft (Blue Angel) 73 degrees of bank (VMC, +Gz Turn). OKCR Head tilt = 31degrees away from the Gz axis.

16 WSU Research Investigating Head Tilt
Patterson (1995, 1997) Smith et al (1997) Merryman et al (1997) Gallimore et al (1999, 2000) Liggett & Gallimore (2001) Gallimore, Liggett & Patterson (2001) Others since

17 OKCR Studies Author Plat-form Visual Field Size Instru-ments VMC Task
OCKR Found? IMC OKCR Found? UA CRE % Subs Patterson (1995) Fixed aircraft sim Full dome 180o HDD AI X Yes No 65% 16 Smith et al. (1997) Merryman et al. (1997) F-15 aircraft Real world HUD 9 Braithwaite et al (1998) Moving Heli-copter Sim Half dome 160o H FOV NVG 25% 20 Gallimore et al. (1999) 31% 12 Gallimore et al. (2000) AII 26

18 Horizon Roll Vs. Head Roll for Low-Level Route
Patterson et al.

19 Three graduate studies: Patterson, Merryman, Smith

20 Merryman & Smith

21 Results: Head tilt with respect to aircraft bank during low-level route Gallimore, et al (1999)

22 OKCR Results

23 OKCR as a function of task and field of view

24 Reversal Error Tendency for pilots to mistake motion of the artificial horizon as a relative motion of the wings. Pilots roll or pitch the aircraft in opposite direction. Researchers who have documented this error Fitts and Jones (1947) Johnson and Roscoe (1972) Roscoe and Williges (1975) Roscoe (1986) - Boeing 747 accident

25 Sensory-Spatial Conflict and Control Reversal Error (Patterson et al findings)
Experienced U.S. military rated pilots commit 25-65% reversal errors. Likelihood of reversal errors by general aviation pilots is probably even greater. A reversal error can lead to flight into terrain or a graveyard spiral. This is likely what happened to the pilot of Air India and to John F. Kennedy, Jr. References on reversal errors: Patterson, et al, 1997 Braithwaite,et al, 1998 Gallimore, et al., (1999) Liggett & Gallimore (in press) Control reversal error during IMC “out” to “in” visual transition. During transition, when the cognitive switch between those spatial strategies switches, pilots make reversal errors. That is, they put in a stick movement opposite to that which they need to level the plane. Here is an example of a military pilot performing a control reversal error in a simulator. In this exercise, the pilot was flying behind a lead aircraft. The lead aircraft was moving into an unusual attitude, and we took away the lead aircraft. The pilot must look at their instruments and return to level flight as quickly as possible when they lose the lead. You will hear the experimenter say “ok take it away” which means the lead is removed. What you see is the attitude indicator go in one direction, reverse, then reverse again and the pilot does a roll to recover. This was not the fastest way to return to wings level.

26 Number and Magnitude of Reversal Errors Gallimore, et al findings
40 Degrees 60 Degrees 100 Degrees 9 errors out of 24 8 errors out of 24 6 errors out of 24 VMC 4(17.39%) 1(.04%) 5(20.83%) 4(17.39%) 5(20.83%) 4(17.39%) IMC Average reversal error magnitude 9.34 o Combined Average reversal error magnitude 28.96 o Average reversal error magnitude 9.30 o

27 Transitions What happens during the transition from visual to instruments? The pilot’s view of the cockpit suddenly becomes stationary as his view of the display’s artificial horizon begins moving. Pilots must instantly reverse their orientation strategy. Pilots sensory-spatial compatibility between the control stick motion and visual feed back.

28 Summary Pilots reflexively tilt heads toward horizon during VMC roll maneuvers. Head movement acts to stabilize retinal image. Generated by motion on retina, not vestibular. Stabilized horizon is the primary visual cue. Peripherally viewed cockpit structures secondary cues. Secondary cues move with airframe. Control movement compatible with secondary cues.

29 Summary (Cont.) Beyond 40 degrees of aircraft roll there is a decrease in head displacement, so pilots can not stabilize the horizon. Horizon acceleration, stabilization of secondary cues. Sudden switch may lead to false perceptions. When transitioning from visual to instruments motion reversal b/w outside and inside visual cues control display incompatibility need to switch cognitive model

30 How does OKCR affect current display technologies?
Head down Attitude Indicator Reversal errors HUD Head may tilt out of the HUD eye box and pilot may not see a pull up X.

31 HUD The Head Up Display (HUD) presents symbols to the pilot, displaying them over the real world.

32 HUD Symbology is Conformal

33 HUD Symbology

34 How does OKCR affect current display technologies? (cont)
NVG HUD symbology on the NVG. Head movements are not tracked. As pilot changes head position, display horizon line is no longer conformal to the real horizon. Pilots see HUD information designed for fixed on-axis aircraft viewing regardless of head position. Pilots may not realize they are not flying in the direction they are looking.

35 Research Issues What frames of reference are important for a pilot to maintain orientation? World - world is fixed and everything moves within it. Aircraft - aircraft is fixed and everything moves around it. Pilot - pilot is fixed and everything moves in relation to him. One of the most fundamental issues of maintaining spatial orientation (SO) is to identify the most desirable frame of reference for establishing a correct attitude in a pilot’s internal model of SO. There have been numerous studies to characterize a visual reflex called the optokinetic cervical reflex (OKCR) that implies that pilots use a world frame of reference to orient themselves when looking at real world visual cues. When pilots use instruments to determine orientation, the information is portrayed in an aircraft frame of reference. This change in frames of reference when pilots transition from visual meteorological conditions (real-world cues) to instrument meteorological conditions (instrument cues) may be directly affecting their SO model, leading to spatial disorientation (SD)

36 Research Issues What symbology is appropriate for HMDs?
HUD symbology is being considered for use on HMDs. HUD symbology is being used on NVGs. How do sensory reflexes affect perceived frame of reference? OKCR, under VMC pilots align their head with the horizon.

37 Research Issues How do visual frames of reference interact with vestibular and proprioceptive inputs to provide the pilot with an "awareness" of their orientation? What contributing cognitive factors affect spatial orientation? How will HMD attitude symbology affect frames of reference in VMC and IMC? How will transitions be impacted? How can we detect when a pilot is spatially disoriented?

38 Research Issues for HMD Symbology Design
What spatial sensory reflexes and visual illusions influence pilot’s perception of frame of reference? Will cognitive capture affect pilots perceptions of frame of reference? Will cognitive capture result in more transitions between symbology and the real world? When pilots transition between a perceived stationary horizon (real world cues) to a moving symbol horizon on the HMD, do they perceive the horizon symbol as stationary? What type of symbology will help provide the perception of a stationary horizon?

39 Research Issues for HMD Symbology Design
If HMD symbology is used for attitude information as well as targeting, how will switching between these tasks affect frame of reference? Will pilots have a greater risk of spatial disorientation if they look off-axis more often? How will secondary flight cues be affected by use of HMDs? What current or new measures should be employed to determine if a pilot is spatially disoriented?

40 HMD Research and the OKCR
Experiment I Test adequacy of Mil-Std HUD symbology presented on a see-through HMD during various tasks. VMC flight task Pilots were instructed to bank at specific angles, rather than to bank around a waypoint. 12 Subjects HMD Kaiser Pro – Binocular HMD, 40o circular FOV, 100% overlap, 1280 x 1024 resolution.

41 HMD Research and the OKCR
Experiment II Investigate visual cues in an immersed HMD simulation system using HUD symbology. VMC Flight task Varied resolution (640 x 480 & 800 x 600), HUD symbol size (small and large) Pilots instructed to follow a yellow track line over Pensacola, FL 6 subjects Virtual Research V8 HMD system, 48o H x 32o V, 100% overlap

42 HMD Research and the OKCR
Experiment III Investigate the effects of non-congruent motion on performance in an immersive HMD system. VMC flight task flown on land and on Navy mind sweeper in Pensacola Bay. Pilots instructed to follow a yellow track line over Pensacola, FL 9 subjects Sony i-glasses , 24o H x 18o V, 100% overlap, 789 x 230 Resolution

43 HMD Results EXP I Analysis of the data and pilot comments indicated that subjects did not tilt their head because they were using the bank scale symbology on the HMD to determine and hold their bank angle prescribed during the task via a verbal command to bank their aircraft to a certain bank angle and to maintain that bank until instructed to level out. Pilots did not have to rely on any outside visual cues to conduct the task resulting in an instrument-oriented task. In all previous studies pilots were flying tasks that required them to maintain awareness of the real-world visual scene in order to bank the aircraft. EXP II There was a significant difference in head tilt as a function of aircraft bank, indicating an OKCR response under VMC as illustrated in Figure 3. While the OKCR response is obvious, the magnitude of the response is much smaller than previous studies using low level flight tasks. The OKCR response is very similar to results found by Gallimore et al. (2000)5 (Figure 2) in which pilots flew at higher altitudes. EXP III As indicated in the figure, both land and shipboard HMD simulations produced significant OKCR responses. The OKCR differences between ship and land conditions were not significant. The results are very similar to results from Experiment II. Preliminarily results indicate that individuals who reported higher scores on their motion sickness surveys exhibited less OKCR during the shipboard simulation compared to the land based condition.

44 OKCR Differences Different visual scenes/cues cause difference in pilot OKCR response Reducing FOV Manipulating altitude Amount of head tilt depends on amount of retinal movement. Reduction in peripheral vision may play a role Reducing FOV may reduce how compelling the visual horizon appears

45 OKCR Differences Immersive HMD simulation studies did not provide any secondary visual cues (cockpit structures). Do pilots reduce head movements when they lack a stabilizing cue? If experiencing simulator sickness may reduce head movements.

46 Control Reversal Errors HMD Liggett and Gallimore findings
Overall CRE rate 28%, similar to previous studies. Magnitude range: 6 degrees to 201 degrees A conformal horizon symbol did not reduce CREs. Because we know they were not tilting in IMC, they still had to change frames of reference from world to aircraft.

47 Control Reversal Errors HMD Liggett and Gallimore findings
Dependent measure: Altitude Change Significant difference CRE group average: ft MSL No CRE group average: ft MSL Pilots with CREs obviously confused. Focusing on pitch and bank information in central part of symbology. Fail to scan airspeed and altitude information.

48 1. Recognition of pilot spatial awareness strategies
3. Avoidance and recognition of Visual Illusions (perspective illusion) 2. Avoidance and recognition of spatial disorientation (VMC-IMC form/ reversal error) 4. Design of flightdeck displays Avoidance and recognition of visual illusions.

49 Spatial disorientation factor Perspective (moon) illusion
For the AVIANO mishap, I don’t have time to go over all the issues, but I would like to show you one factor that affected the pilot’s spatial awareness. This information comes from expert witnesses at the trial. This is an example of the Perspective (or Moon) illusion. Some of you may have experienced it before. When the moon is low and just over the horizon it appears to be very large. But if you took a picture, it would be the “normal” size. In this illusion, the box in the back looks larger than the one in the front, yet they are actually the same size. Things on the horizon are thrown out of perspective. Even it you know it is an illusion, you can’t turn it off. It can’t be trained away. It is very compelling.

50 Example: Perspective Illusion
This is a tape of the valley in AVIANO where the mishap took place. This tape was recorded by flying a helicopter over the valley at the same altitude that the plane flew. The prosecution recorded this tape. The cables were gone, and they put up two white weather balloons to show where he cables were located. Some basic information. The pilot was practicing a low level flight. He must keep his view outside the aircraft. It is mountainous terrain. It only takes 30 seconds to get from one end of the valley to the next. The mishap happens about 20 seconds into the valley. This tape is from a point in the valley when there is only 12 seconds before the pilot (flying 500 knots 900 feet per sec) reached the mishap site. The helicopter is flying at 400 knots.

51 This is the same tape. But on it is a line which indicates the horizontal reference that the pilot would be fixating on or looking at in order to fly through the valley. The balloons are highlighted in red so you can see them. What do you see? As you start out it appears that the balloons are below the horizontal reference point, but as we proceed the balloons appear to rise. Are you below the balloons? You experienced the perspective illusion. The red balloons are not rising. The horizontal reference point is changing. Which you can see if I put my cursor at the place were we see the red ball, then when it is finished they are still level with my cursor. At all times the plane is actually 100 feet above the balloons. We never go below them. But to the pilot it appears as if you are going to hit the balloons. He sees the first cable, thinks he is going to hit it. He has a few seconds to respond, he pushes the nose down to go under it because the aircraft can’t go above in that time period, he missing the first cable, but hits the second on that is lower than the first.

52 References Aviation Research
Jenkins, J. C., and Gallimore, J.J. (2008). Configural display design features to promote pilot situation awareness in helmet-mounted displays. Aviation, Space and Environmental Medicine, 79, Stephens, M., Gallimore, J., and Albery, W. (2002) Spectral Analysis of Electroencephalographic Response to Spatial Disorientation. Proceedings of the 12th International Symposium on Aviation Psychology: Dayton OH. (pp ). Liggett, K.K. and Gallimore, J.J. (2002). The effects of frame of reference and HMD symbology on control reversal errors. Aviation, Space, and Environmental Medicine;73: Gallimore, J.J., Liggett, K.K. and Patterson, F.R. (2001). The Opto-Kinetic Cervical Reflex in Flight Simulation. Proceedings of the American Institute of Aeronautics and Astronautics Modeling and Simulation Conference and Exhibit, Aug 6-9, 2001, Montreal, Canada, Paper No: : pp 1-7. * Best Paper.

53 References Aviation Research
Liggett, K. and Gallimore, J.J. (2001) The OKCR and Pilot Performance During Transitions Between Meteorological Conditions Using HMD Attitude Symbology. In Proceedings of the Human Factors and Ergonomics Society 45th Annual Meeting, (pp ) Santa Monica. CA HFES. Gallimore, J.J., Patterson, F.R., Brannon, N.G., and Nalepka, J.P. (2000). The opto-kinetic cervical reflex during formation flight. Aviation, Space and Environmental Medicine 2000;71: Gallimore J. J., Brannon, N. G., Patterson, F.R., and Nalepka, J.P. (1999). Effects of FOV and aircraft bank on pilot head movement and reversal errors during simulated flight. Aviation, Space and Environmental Medicine, 70(12): Gallimore, J.J., Brannon, N.G., and Patterson F.R. (1998). The Effects of Field-of-View on Pilot Head Movement During Low Level Flight. In Proceedings of the Human Factors and Ergonomics Society 42nd Annual Meeting, Chicago, IL (pp. 6-10). Patterson F. R., Cacioppo, A. J., Gallimore, J.J., Hinman, G.E., and Nalepka, J.P. (1997). Aviation spatial orientation in relationship to head position and attitude interpretation. Aviation, Space and Environmental Medicine, 68(6),

54 Other Research A Predictive Model Of Cognitive Performance Under Acceleration Stress Submitted to Aviation, Space, Environmental Medicine, June 09 Three-Dimensional Technology for Space Operation Applications Multi-modal Displays for Portraying Meta-Info to Support Net-Centric C2 Process Control Displays Virtual Patients Collaborative Computer Agents with Personality

55 Acknowledgements CDR Frederick Patterson, Ph.D., Retired
Naval Aerospace Medical Research Laboratory United States Navy I would like to acknowledge Cmd Fred Patterson of the Naval Aeromedical Research Laboratory, who has funded some of this research and has provided some materials for this presentation.

56 Thank You

Download ppt "Jennie J. Gallimore, Ph.D. June 24, 2009 NASA Langley"

Similar presentations

Ads by Google