Presentation is loading. Please wait.

Presentation is loading. Please wait.

Noise and Echo Control for Immersive Voice Communication in Spacesuits 9/2/2010 Yiteng (Arden) Huang WeVoice, Inc., Bridgewater, New Jersey, USA

Similar presentations


Presentation on theme: "Noise and Echo Control for Immersive Voice Communication in Spacesuits 9/2/2010 Yiteng (Arden) Huang WeVoice, Inc., Bridgewater, New Jersey, USA"— Presentation transcript:

1 Noise and Echo Control for Immersive Voice Communication in Spacesuits 9/2/2010 Yiteng (Arden) Huang WeVoice, Inc., Bridgewater, New Jersey, USA Presented as a keynote speech on the International Workshop on Acoustic Echo and Noise Control (IWAENC) in Tel Aviv, Israel on September 2, 2010

2 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits About the Project  Financially sponsored by the NASA SBIR (Small Business Innovation Research) program  Phase I feasibility research: Jan – July 2008  Phase II prototype development: Jan – Jan  Other team members: Jingdong Chen, WeVoice, Inc., Bridgewater, New Jersey, USA Scott Sands, NASA Glenn Research Center (GRC), Cleveland, Ohio, USA Jacob Benesty, University of Quebec, Montreal, Quebec, Canada  Financially sponsored by the NASA SBIR (Small Business Innovation Research) program  Phase I feasibility research: Jan – July 2008  Phase II prototype development: Jan – Jan  Other team members: Jingdong Chen, WeVoice, Inc., Bridgewater, New Jersey, USA Scott Sands, NASA Glenn Research Center (GRC), Cleveland, Ohio, USA Jacob Benesty, University of Quebec, Montreal, Quebec, Canada

3 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits Outline 1.Problem Identification and Research Motivation 2.Problem Analysis and Technical Challenges 3.Noise Control with Microphone Arrays 4.Hardware Development 5.Software Development 6.A Portable, Real-Time Demonstration System 7.Towards Immersive Voice Communication in Spacesuits 1.Problem Identification and Research Motivation 2.Problem Analysis and Technical Challenges 3.Noise Control with Microphone Arrays 4.Hardware Development 5.Software Development 6.A Portable, Real-Time Demonstration System 7.Towards Immersive Voice Communication in Spacesuits

4 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits Section 1 1.Problem Identification and Research Motivation 2.Problem Analysis and Technical Challenges 3.Noise Control with Microphone Arrays 4.Hardware Development 5.Software Development 6.A Portable, Real-Time Demonstration System 7.Towards Immersive Voice Communication in Spacesuits 1.Problem Identification and Research Motivation 2.Problem Analysis and Technical Challenges 3.Noise Control with Microphone Arrays 4.Hardware Development 5.Software Development 6.A Portable, Real-Time Demonstration System 7.Towards Immersive Voice Communication in Spacesuits

5 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits Requirements of In-Suit Audio  Speech Quality and Intelligibility:  90% word identification rate  Hearing Protection:  Limits total noise dose, hazard noise, and on-orbit continuous and impulse noise for waking and sleeping periods  Noise loads are very high during launch and orbital maneuvers.  Audio Control and Interfaces:  Provides manual silencing features and volume controls  Operation at Non-Standard Barometric Pressure Levels (BPLs):  Operates effectively between 30 kPa and 105 kPa  Speech Quality and Intelligibility:  90% word identification rate  Hearing Protection:  Limits total noise dose, hazard noise, and on-orbit continuous and impulse noise for waking and sleeping periods  Noise loads are very high during launch and orbital maneuvers.  Audio Control and Interfaces:  Provides manual silencing features and volume controls  Operation at Non-Standard Barometric Pressure Levels (BPLs):  Operates effectively between 30 kPa and 105 kPa

6 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits Current In-Suit Audio System Chin Cup Microphone Module Microphone Boom Skullcap Perspiration Absorption Area Helmet Helmet Ring Earpiece Current Solution: Communication Carrier Assembly (CCA) Audio System

7 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits Extravehicular Mobility Unit (EMU) CCA For shuttle and International Space Station (ISS) operations Source: O. Sands, NASA GRC Interconnect wiringNylon/spondex top Teflon sidepiece and pocket Electret Microphone Interface cable and connector Electret Microphone Ear seal Ear cup A large gain applied to the outbound speech for sufficient sound volume at low static pressure levels (30 kPa) leads to clipping and strong distortion during operations near sea-level BPL.

8 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits Advanced Crew Escape Suit (ACES) CCA Source: O. Sands, NASA GRC Dynamic Microphones For shuttle launch and entry operations Hearing protection provided by the ACES CCA may not be sufficient.

9 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits Developmental CCA The active earpieces will be used in conjunction with the CCA ear cups during launch and other high noise events and can be removed for other suited operations. The active earpieces alone nearly provide the required level of hearing protection. The active earpieces will be used in conjunction with the CCA ear cups during launch and other high noise events and can be removed for other suited operations. The active earpieces alone nearly provide the required level of hearing protection. Noise Canceling Microphones Active In-Canal Earpieces Source: O. Sands, NASA GRC Ear Cups

10 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits CCA Systems: Pros High outbound speech intelligibility and quality, SNR near optimum  Use close-talking microphones  A high degree of acoustic isolation between the in-suit noise and the suit subject’s vocalizations  A high degree of acoustic isolation between the inbound and outbound signals  The human body does NOT transmit vibration-borne noise Provide very good hearing protection. High outbound speech intelligibility and quality, SNR near optimum  Use close-talking microphones  A high degree of acoustic isolation between the in-suit noise and the suit subject’s vocalizations  A high degree of acoustic isolation between the inbound and outbound signals  The human body does NOT transmit vibration-borne noise Provide very good hearing protection.

11 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits The microphones need to be close to the mouth of a suited subject. A number of recognized logistical issues and inconveniences:  Cannot adjust the cap and the microphone booms during EVA operations, which can last from 4 to 8 hours  The close-talking microphones interfere with the suited subject’s eating and drinking, and are susceptible to contamination.  The communication cap needs to fit well. Caps in a variety of different sizes need to be built and maintained, e.g., 5 sizes for EMU caps.  Wire fatigue for the microphone booms These problems cannot be resolved with incremental improvements to the basic design of the CCA systems. The microphones need to be close to the mouth of a suited subject. A number of recognized logistical issues and inconveniences:  Cannot adjust the cap and the microphone booms during EVA operations, which can last from 4 to 8 hours  The close-talking microphones interfere with the suited subject’s eating and drinking, and are susceptible to contamination.  The communication cap needs to fit well. Caps in a variety of different sizes need to be built and maintained, e.g., 5 sizes for EMU caps.  Wire fatigue for the microphone booms These problems cannot be resolved with incremental improvements to the basic design of the CCA systems. CCA Systems: Cons

12 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits Stakeholder Interviews The CCA ear cups produce pressure points that cause discomfort. Microphone arrays and helmet speakers are suggested to be used. Suit subject comfort should be maximized as much as possible, given that other constraints can be met (relaxed and traded off):  Clear two-way voice communications  Hearing protection from the fan noise in the life support system ventilation loop  Properly containing and managing hair and sweat inside the helmet  Adequate SNR for the potential use of automatic speech recognition for the suit’s information system The CCA ear cups produce pressure points that cause discomfort. Microphone arrays and helmet speakers are suggested to be used. Suit subject comfort should be maximized as much as possible, given that other constraints can be met (relaxed and traded off):  Clear two-way voice communications  Hearing protection from the fan noise in the life support system ventilation loop  Properly containing and managing hair and sweat inside the helmet  Adequate SNR for the potential use of automatic speech recognition for the suit’s information system

13 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits Two Alternative Architectural Options for In-Suit Audio 1.Integrated Audio (IA): Instead of being housed in a separate subassembly, both the microphones and the speakers are integrated into the suit/helmet. 2.Hybrid Approach: Employs the inbound portion of a CCA system with the outbound portion of an IA system. 1.Integrated Audio (IA): Instead of being housed in a separate subassembly, both the microphones and the speakers are integrated into the suit/helmet. 2.Hybrid Approach: Employs the inbound portion of a CCA system with the outbound portion of an IA system. Helmet Speaker

14 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits Section 2 1.Problem Identification and Research Motivation 2.Problem Analysis and Technical Challenges 3.Noise Control with Microphone Arrays 4.Hardware Development 5.Software Development 6.A Portable, Real-Time Demonstration System 7.Towards Immersive Voice Communication in Spacesuits 1.Problem Identification and Research Motivation 2.Problem Analysis and Technical Challenges 3.Noise Control with Microphone Arrays 4.Hardware Development 5.Software Development 6.A Portable, Real-Time Demonstration System 7.Towards Immersive Voice Communication in Spacesuits

15 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits Noise from Outside the Spacesuit During launch, entry descent, and landing:  Impulse noise < 140 dBSPL, Hazard noise < 105 dBA On orbit:  Impulse noise: < 140 dBSPL waking hours and < 83 dBSPL sleeping  Limits on continuous on-orbit noise levels by frequency: Remark: During EVA operations, ambient noise is at most a minor problem. During launch, entry descent, and landing:  Impulse noise < 140 dBSPL, Hazard noise < 105 dBA On orbit:  Impulse noise: < 140 dBSPL waking hours and < 83 dBSPL sleeping  Limits on continuous on-orbit noise levels by frequency: Remark: During EVA operations, ambient noise is at most a minor problem. Band Center Frequency (Hz) k2k4k8k16k Sound Pressure Level (dB) SPL (dB)85 – 9575 – 8565 – 7555 – 65 Perception Very High Noise: speech almost impossible to hear High Noise: speech is difficult to hear Medium Noise: Must Raise Voice to be Heard Low Noise: speech is easy to hear Typical Environments Construction Site Loud Machine Shop Noisy Manufacturing Assembly Line Crowded Bus/Transit Waiting Area Very Noisy Restaurant/Bar Department Store Band/Public Area Supermarket Doctor’s Office Hospital Hotel Lobby

16 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits Structure-Borne Noise Inside the Spacesuit Four noise sources (Begault & Hieronymus 2007): 1.Airflow and air inlet hissing noise, as well as fan/pump noise due to required air supply and circulation 2.Arm, leg, and hip bearing noise 3.Suit-impact noise, e.g., footfall 4.Swishing-like noise due to air movement caused by walking (since the suits are closed pressure environments) Four noise sources (Begault & Hieronymus 2007): 1.Airflow and air inlet hissing noise, as well as fan/pump noise due to required air supply and circulation 2.Arm, leg, and hip bearing noise 3.Suit-impact noise, e.g., footfall 4.Swishing-like noise due to air movement caused by walking (since the suits are closed pressure environments) For CCA systems, since the suit subject’s body does not transmit bearing and impact noise, only airflow-related noise needs to be controlled. For Integrated Audio (IA) systems, microphones are mounted directly on the suit structure and vibration noise is loud. For CCA systems, since the suit subject’s body does not transmit bearing and impact noise, only airflow-related noise needs to be controlled. For Integrated Audio (IA) systems, microphones are mounted directly on the suit structure and vibration noise is loud.

17 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits Acoustic Challenges Complicated noise field:  Temporal domain:Has both stationary and non-stationary noise  Spectral domain:Inherently wideband  Spatial domain:Near field; Possibly either directional or dispersive Highly reverberant enclosure:  The helmet is made of highly reflective materials.  Strong reverberation dramatically reduces the intelligibility of speech uttered by the suit subject and degrades the performance of an automatic speech recognizer.  Strong reverberation leads to a more dispersive noise field, which makes beamforming less effective.

18 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits Section 3 1.Problem Identification and Research Motivation 2.Problem Analysis and Technical Challenges 3.Noise Control with Microphone Arrays 4.Hardware Development 5.Software Development 6.A Portable, Real-Time Demonstration System 7.Towards Immersive Voice Communication in Spacesuits 1.Problem Identification and Research Motivation 2.Problem Analysis and Technical Challenges 3.Noise Control with Microphone Arrays 4.Hardware Development 5.Software Development 6.A Portable, Real-Time Demonstration System 7.Towards Immersive Voice Communication in Spacesuits

19 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits Proposed Noise Control Scheme for IA/Hybrid Systems 5 Adaptive Noise Cancellation Beamforming Multichannel Noise Reduction Acoustic Source Localization Head Position Calibration Head Motion Tracker Single Channel Noise Reduction Outbound Speech Mouth range and incident angle with respect to the microphone array Noise Reference Microphone Array

20 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits Current Research Focus 4321 Beamforming Multichannel Noise Reduction Single Channel Noise Reduction Outbound Speech Microphone Array

21 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits Beamforming: Far-Field vs. Near-Field... d θ  hNhN  h2h2  h1h1 Σ Y(f, ψ, r s ) X N (f)X 2 (f)X 1 (f) ψ Far-Field Noise Plane Waves … V(f, ψ) S(f, r s ) Near-Field Sound Source rsrs 12N... … d ψ (N-1) · d·cos(ψ ) Plane Waves θ...  hNhN  h2h2  h1h1 Σ Y(f, ψ, θ) X N (f)X 2 (f)X 1 (f) … S(f, θ) V(f, ψ) Far-Field NoiseFar-Field Sound Source of Interest 12N

22 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits Fixed Beamformer vs. Adaptive Beamformer Microphone Array Beamformers Fixed Beamformers Adaptive Beamformers Delay-and-Sum Filter-and-Sum MVDR (Capon) LCMV (Frost)/GSC Noise Field? Stationary, Known before the designTime Varying, Unknown Isotropic noise generally assumed Reverberation? Not ConcernedSignificant Delay-and-Sum Simple Non-uniform directional responses over a wide spectrum of frequencies Delay-and-Sum Simple Non-uniform directional responses over a wide spectrum of frequencies Filter-and-Sum Complicated Uniform directional responses over a wide spectrum of frequencies: good for wideband signals, like speech Filter-and-Sum Complicated Uniform directional responses over a wide spectrum of frequencies: good for wideband signals, like speech MVDR (Capon) Only the TDOAs of the interested speech source need to be known – simple requirements. Reverberation causes the signal cancellation problem. Time-domain or frequency-domain MVDR (Capon) Only the TDOAs of the interested speech source need to be known – simple requirements. Reverberation causes the signal cancellation problem. Time-domain or frequency-domain LCMV (Frost)/GSC The impulse responses (IRs) from the source to the microphones have to be known or estimated. Errors in the IRs lead to the signal cancellation problem. LCMV (Frost)/GSC The impulse responses (IRs) from the source to the microphones have to be known or estimated. Errors in the IRs lead to the signal cancellation problem.

23 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits Comments on Traditional Microphone Array Beamforming For incoherent noise sources, the gain in SNR is low if the number of microphones is small. For coherent noise sources whose directions are different from that of the speech source, a theoretically optimal gain in SNR can be high but is difficult to obtain due to a number of practical limitations:  Unavailability of precise a priori knowledge of the acoustic impulse responses from the speech sources to the microphones.  Inconsistent responses of the microphones across the array. For coherent noise sources that are in the same direction as the speech source, beamforming (as a spatial filter) is ineffective. For incoherent noise sources, the gain in SNR is low if the number of microphones is small. For coherent noise sources whose directions are different from that of the speech source, a theoretically optimal gain in SNR can be high but is difficult to obtain due to a number of practical limitations:  Unavailability of precise a priori knowledge of the acoustic impulse responses from the speech sources to the microphones.  Inconsistent responses of the microphones across the array. For coherent noise sources that are in the same direction as the speech source, beamforming (as a spatial filter) is ineffective.

24 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits Multichannel Noise Reduction (MCNR) x 1,s (k) Only Denoising... MCNR xN(k)xN(k)x2(k)x2(k)x1(k)x1(k) s(k)s(k) 12N v(k)v(k)... gNgN g2g2 g1g1 Beamformer: Spatial Filtering Array Setup: Calibration is necessary – possibly time/effort consuming Beamformer: Spatial Filtering Array Setup: Calibration is necessary – possibly time/effort consuming MCNR: Statistical Filtering Array Setup: No need to strictly demand a specific array geometry/pattern MCNR: Statistical Filtering Array Setup: No need to strictly demand a specific array geometry/pattern A conceptual comparison of beamforming and MCNR: s(k)s(k)... d Beamforming xN(k)xN(k)x2(k)x2(k)x1(k)x1(k) s(k)s(k) Speech Source of Interest 12N Noise v(k)v(k)... Impulse Responses gNgN g2g2 g1g1 Dereverberation and Denoising Knowledge related to the source position or g n Signal Model:

25 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits Frequency-Domain MVDR Filter for MCNR The problem formulation: The MVDR filter: A more practical implementation: where Similar to traditional single-channel noise reduction methods, the noise PSD matrix is estimated during silent periods and the signal PSD matrixis estimated during speech periods. The problem formulation: The MVDR filter: A more practical implementation: where Similar to traditional single-channel noise reduction methods, the noise PSD matrix is estimated during silent periods and the signal PSD matrixis estimated during speech periods.

26 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits Comparison of the MVDR Filters for Beamforming and MCNR Note: In the implementation of the MVDR-MCNR, the channel responses do not need to be known. The acoustic impulse responses can at best be estimated up to a scale: where denotes the true response vector. Leads to speech distortion. MVDR for MCNR: MVDR for Beamforming (BF):

27 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits Distortionless Multichannel Wiener Filter for MCNR Use what we called the spatial prediction: Formulate the following optimization problem: where The distortionless multichannel Wiener (DW) filter for MCNR: The optimal Wiener solution for the non-causal spatial prediction filters: whereSo, It was found that Use what we called the spatial prediction: Formulate the following optimization problem: where The distortionless multichannel Wiener (DW) filter for MCNR: The optimal Wiener solution for the non-causal spatial prediction filters: whereSo, It was found that

28 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits Single-Channel Noise Reduction (SCNR) for Post-Filtering Beamforming: The Wiener filter (the optimal solution in the MMSE sense) can be factorized as MVDR Beamformer Wiener Filter for SCNR MCNR: Again, the Wiener filter can be factorized as Note: For a complete and detailed development of this factorization, please refer to Eq. (6.117) of the following book.  J. Benesty, J. Chen, and Y. Huang, Microphone Array Signal Processing, Berlin, Germany: Springer, Note: For a complete and detailed development of this factorization, please refer to Eq. (6.117) of the following book.  J. Benesty, J. Chen, and Y. Huang, Microphone Array Signal Processing, Berlin, Germany: Springer, MVDR for MCNR Wiener Filter for SCNR Note: For a complete and detailed development of this factorization, please refer to Eq. (3.19) of the following book.  M. Brandstein and D. Ward, eds, Microphone Arrays: Signal Processing Techniques and Applications, Berlin, Germany: Sprinter, Note: For a complete and detailed development of this factorization, please refer to Eq. (3.19) of the following book.  M. Brandstein and D. Ward, eds, Microphone Arrays: Signal Processing Techniques and Applications, Berlin, Germany: Sprinter, 2001.

29 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits Single-Channel Noise Reduction (SCNR) The signal model: SCNR filter: Error signal: MSE cost function: The Wiener filter: where and Other SCNR methods: Parametric Wiener filter, Tradeoff filter. The signal model: SCNR filter: Error signal: MSE cost function: The Wiener filter: where and Other SCNR methods: Parametric Wiener filter, Tradeoff filter. A well-known feature: Noise reduction is achieved at the cost of adding speech distortion.

30 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits New Idea for SCNR A second-order complex circular random variable (CCRV) has: which implies that and its conjugate are uncorrelated. In general, speech is not a second-order CCRV: But noise is a second-order CCRV if stationary, and not otherwise. A second-order complex circular random variable (CCRV) has: which implies that and its conjugate are uncorrelated. In general, speech is not a second-order CCRV: But noise is a second-order CCRV if stationary, and not otherwise. Examine This is similar to the signal model of a two-element microphone array. So there is a chance to reduce noise without adding any speech distortion. Examine This is similar to the signal model of a two-element microphone array. So there is a chance to reduce noise without adding any speech distortion. Correlated but not completely coherentUncorrelated or coherent

31 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits Widely Linear Wiener Filter New filter for SCNR: Error signal: Widely linear MSE: Then the widely linear Wiener filter or MVDR type of filters can be developed. New filter for SCNR: Error signal: Widely linear MSE: Then the widely linear Wiener filter or MVDR type of filters can be developed.

32 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits Section 4 1.Problem Identification and Research Motivation 2.Problem Analysis and Technical Challenges 3.Noise Control with Microphone Arrays 4.Hardware Development 5.Software Development 6.A Portable, Real-Time Demonstration System 7.Towards Immersive Voice Communication in Spacesuits 1.Problem Identification and Research Motivation 2.Problem Analysis and Technical Challenges 3.Noise Control with Microphone Arrays 4.Hardware Development 5.Software Development 6.A Portable, Real-Time Demonstration System 7.Towards Immersive Voice Communication in Spacesuits

33 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits Computational Platform/Technology Selection  Three platforms under consideration: ASIC DSP FPGA  Trade-off among performance, power consumption, size, and costs  Three platforms under consideration: ASIC DSP FPGA  Trade-off among performance, power consumption, size, and costs  Four competing factors: The count of transistors employed The number of clock cycles required The time taken to develop an application Nonrecurring engineering (NRE) costs  Four competing factors: The count of transistors employed The number of clock cycles required The time taken to develop an application Nonrecurring engineering (NRE) costs ASIC Low numbers of transistors and clock cycles Long development time and high NRE costs Effective in performance, power, and size, but not in cost ASIC Low numbers of transistors and clock cycles Long development time and high NRE costs Effective in performance, power, and size, but not in cost DSP Low development and NRE costs Low power consumption More efforts to convert the design to ASICs DSP Low development and NRE costs Low power consumption More efforts to convert the design to ASICs FPGA Not suited to processing sequential conditional data flow, but efficient in concurrent applications Support faster I/O than DSPs One step closer to ASIC than DSP High development cost due to performance optimization FPGA Not suited to processing sequential conditional data flow, but efficient in concurrent applications Support faster I/O than DSPs One step closer to ASIC than DSP High development cost due to performance optimization

34 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits Mic. Powering Circuit GND HOT COLD System Block Diagram DB25 Female XLR Female XLR Male MIC CAPSULE DB25 Male FPGA Board Mic. Powering Circuit GND HOT COLD ch 24-bit 48kHz ADC 8-ch 24-bit 48kHz ADC Mic. Preamps G G G G G G G G Jumpers (for Gain Control) Altera FPGA Altera FPGA JTAG (Male) SDRAM Digital Output Interface (USB 2.0) Power Mgmt IC Power Jack Analog Input Flash Mic. Powering Circuit GND HOT COLD Mic. Powering Circuit GND HOT COLD Mic. Powering Circuit GND HOT COLD Mic. Powering Circuit GND HOT COLD Mic. Powering Circuit GND HOT COLD Mic. Powering Circuit GND HOT COLD 3 2 1

35 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits FPGA Board Block Diagram OPA1632 (1) OPA1632 (2) OPA1632 (8) ADS1278 EPCS16 Altera Cyclone III EP3C55F484C8 FPGA Altera Cyclone III EP3C55F484C8 FPGA 16 MB SDRAM (×32) 16 MB Flash (×16) 50 MHz XTAL MHz XTAL USB 2.0 (High Speed) User LED/IOs 3.3 V

36 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits Prototype FPGA Board: the Top View Phantom Power Feeding Mic. Pream Gain Jumpers OPA1632 REF1004ADS1278 User LEDs EPCS16 S User I/Os JTAG FT2232H USB 2.0 Jack 12 MHz Crystal GND TPS65053 Flash DC Power Jack Power LED SDRAMs Cyclone III FPGA Analog Power DC 9V Analog Power DC 5V DB mm × 101 mm

37 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits Prototype FPGA Board: the Bottom View OPA MHz Clock Oscillator (OSC2) MHz Clock Oscillator (OSC1)

38 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits FPGA System Development Flow Adopted in the Project System on Programmable Chip (SoPC) + C/C++ Programming: 1)Use SoPC Builder to construct a soft-core NIOS II processor embedded on the Altera FPGA 2)Develop software/DSP systems in C/C++ on the NIOS II processor System on Programmable Chip (SoPC) + C/C++ Programming: 1)Use SoPC Builder to construct a soft-core NIOS II processor embedded on the Altera FPGA 2)Develop software/DSP systems in C/C++ on the NIOS II processor Advantages: Short development cycle/time Low cost High reliability Reusability of intellectual property Advantages: Short development cycle/time Low cost High reliability Reusability of intellectual property Drawbacks:  Poor efficiency and low performance:  Efficiency can be improved by identifying those time-consuming functions (e.g., FFT and IFFT) and accelerating them with the tool of C2H (C-to-Hardware) Drawbacks:  Poor efficiency and low performance:  Efficiency can be improved by identifying those time-consuming functions (e.g., FFT and IFFT) and accelerating them with the tool of C2H (C-to-Hardware) CPU (NIOS II) ROMRAM I/O UARTDSP

39 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits a b d ca b d ca b d ca b d ca b d ca b d ca b d c Analog Device ADMP402 MEMS Microphones: 2.5 mm × 3.35 mm mm 20 mm 7 Subarrays Pin 18 Pin 1 XG-MPC-MEMS MEMS Microphone Array Samsung 18-pin Connector

40 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits MEMS Microphone Array Box Pin 1 Pin 18 Samsung 18-pin Connector Wevoice MEMS Microphone Array mm 12.5 mm 155 mm

41 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits Section 5 1.Problem Identification and Research Motivation 2.Problem Analysis and Technical Challenges 3.Noise Control with Microphone Arrays 4.Hardware Development 5.Software Development 6.A Portable, Real-Time Demonstration System 7.Towards Immersive Voice Communication in Spacesuits 1.Problem Identification and Research Motivation 2.Problem Analysis and Technical Challenges 3.Noise Control with Microphone Arrays 4.Hardware Development 5.Software Development 6.A Portable, Real-Time Demonstration System 7.Towards Immersive Voice Communication in Spacesuits

42 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits FPGA Program Flowchart data in & preprocessing MCNR+SCNR 4-ch FFT 1-ch IFFT overlap add USB trans. data in & preprocessing MCNR+SCNR 4-ch FFT 1-ch IFFT overlap add USB trans. time (ms) tt+4t+8 1 time frame Nios II Soft Core FFT/IFFT Processor To USB From ADC FPGA Processing delay < 8 ms

43 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits IA System Windows Host Software Programmed with Microsoft Visual C++ Direct Sound is used to play back audio (speech). Programmed with Microsoft Visual C++ Direct Sound is used to play back audio (speech). Splash window of the program

44 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits IA System Windows Host GUI: Multitrack View

45 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits IA System Windows Host GUI: Single-Track View

46 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits IA System Windows Host GUI: Playing Back

47 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits Section 6 1.Problem Identification and Research Motivation 2.Problem Analysis and Technical Challenges 3.Noise Control with Microphone Arrays 4.Hardware Development 5.Software Development 6.A Portable, Real-Time Demonstration System 7.Towards Immersive Voice Communication in Spacesuits 1.Problem Identification and Research Motivation 2.Problem Analysis and Technical Challenges 3.Noise Control with Microphone Arrays 4.Hardware Development 5.Software Development 6.A Portable, Real-Time Demonstration System 7.Towards Immersive Voice Communication in Spacesuits

48 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits The Portable, Real-Time Demo System FPGA Board Power Supply: Linear DC 12-20V/1A Suited Subject DB25 Connectors PC USB 2.0 Cable MEMS Microphone Array Audio Cable

49 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits Section 7 1.Problem Identification and Research Motivation 2.Problem Analysis and Technical Challenges 3.Noise Control with Microphone Arrays 4.Hardware Development 5.Software Development 6.A Portable, Real-Time Demonstration System 7.Towards Immersive Voice Communication in Spacesuits 1.Problem Identification and Research Motivation 2.Problem Analysis and Technical Challenges 3.Noise Control with Microphone Arrays 4.Hardware Development 5.Software Development 6.A Portable, Real-Time Demonstration System 7.Towards Immersive Voice Communication in Spacesuits

50 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits What is and Why do we want Immersive Communication?  Telecommunication helps people collaborate and share information by cutting across the following 3 separations/constraints:  Long distance  Real time  Physical boundaries  Modern telecommunication technologies are successful so far in transcending the first two constraints: i.e., the long-distance and real-time constraints.  Immersive communication offers an feeling of being together and sharing a common environment during collaboration.  Immersive communication targets at breaking the physical boundaries, which is the “last mile” problem in communication.  Telecommunication helps people collaborate and share information by cutting across the following 3 separations/constraints:  Long distance  Real time  Physical boundaries  Modern telecommunication technologies are successful so far in transcending the first two constraints: i.e., the long-distance and real-time constraints.  Immersive communication offers an feeling of being together and sharing a common environment during collaboration.  Immersive communication targets at breaking the physical boundaries, which is the “last mile” problem in communication.

51 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits What need to be solved for immersive communication systems? Single-Channel Acoustic Echo Cancellation

52 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits What need to be solved for immersive communication systems? Multichannel Acoustic Echo Cancellation

53 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits Synthesized Stereo Audio Mixing System What need to be solved for immersive communication systems?

54 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits What need to be solved for immersive communication systems? Beamforming Blind Source Separation

55 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits What need to be solved for immersive communication systems? Acoustic Source Localization and Tracking

56 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits What need to be solved for immersive communication systems? Stereophony System for Spatial Sound Reproduction

57 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits What need to be solved for immersive communication systems? Wave Field Synthesis

58 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits Why Immersive Voice Communication in Spacesuits?  Immersive voice communication exploits human’s binaural hearing.  Provides enhanced situational awareness for a suited crewmember: Can improve the productivity of collaboration among the crewmembers Can produce potential safety benefits  Crew comfort can be optimized.  Immersive voice communication exploits human’s binaural hearing.  Provides enhanced situational awareness for a suited crewmember: Can improve the productivity of collaboration among the crewmembers Can produce potential safety benefits  Crew comfort can be optimized.

59 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits What Problems Need to be Solved? Stereo/Multichannel Acoustic Echo Cancellation (MCAEC) Integration of MCAEC and MCNR Three Dimensional (3D) Audio Stereo/Multichannel Acoustic Echo Cancellation (MCAEC) Integration of MCAEC and MCNR Three Dimensional (3D) Audio

60 All Rights Reserved © WeVoice, Inc | Huang: Noise and Echo Control for Immersive Voice Communication in Spacesuits Conclusions While it has been more than 40 years since Neil Armstrong landed on the Moon, the astronauts are still using the communication carrier assembly (CCA) based audio system for voice communication in spacesuits. The new spacesuit design is going to take advantage of the most recent advances in multichannel acoustic and speech signal processing for echo and noise control and meanwhile with significantly improved crew comfort and ease of use.  Noise reduction with microphone arrays  Multichannel echo cancellation  Integrated echo and noise control  3D audio We explained the difference between the traditional beamforming method and what we called the multichannel noise reduction approach. We presented an intuitive interpretation of the widely linear Wiener filter for single- channel noise reduction. We described a new application of immersive communication in space exploration, ancillary to its mainstream use in commercial telecommunication systems. While it has been more than 40 years since Neil Armstrong landed on the Moon, the astronauts are still using the communication carrier assembly (CCA) based audio system for voice communication in spacesuits. The new spacesuit design is going to take advantage of the most recent advances in multichannel acoustic and speech signal processing for echo and noise control and meanwhile with significantly improved crew comfort and ease of use.  Noise reduction with microphone arrays  Multichannel echo cancellation  Integrated echo and noise control  3D audio We explained the difference between the traditional beamforming method and what we called the multichannel noise reduction approach. We presented an intuitive interpretation of the widely linear Wiener filter for single- channel noise reduction. We described a new application of immersive communication in space exploration, ancillary to its mainstream use in commercial telecommunication systems.


Download ppt "Noise and Echo Control for Immersive Voice Communication in Spacesuits 9/2/2010 Yiteng (Arden) Huang WeVoice, Inc., Bridgewater, New Jersey, USA"

Similar presentations


Ads by Google