1. Nonlinear MMSE Estimation and Soft Decisions Stefano Galli Dr. Stefano Galli Telcordia Technologies, Inc. Room: MCC-1J124B 445 South Street Morristown, NJ 07960-6438 Tel.: (973) 829-4980 Fax: (973) 829-5886 Email: sgalli@research.telcordia.com Copyright © 2003 Telcordia Technologies. All Rights Reserved John Hopkins University, April 17, 2003.

2. Telcordia Technologies Proprietary - Copyright 2003. Telcordia Technologies Applied Research Core Competencies  Network and Services Management –Services Planning and Provisioning –Broadband Network Management –Wireless Network Management –Global Services Management  Software Technology and Engineering –High Availability Software –Distributed Systems –Scalable Systems –Software Architecture and Testing

3. Telcordia Technologies Proprietary - Copyright 2003.  Internet Evolution –Internet Architecture/Performance/Economics –Quality of Service (QoS) in Converged Networks  Next-Generation Networks –Voice over Packet (VoP)/Voice over IP (VoIP) –Multi-Protocol Label Switching (MPLS) –Integrated Access Networking –Internet Appliances and Premises Interworking –Next-Generation Signaling and Control Telcordia Technologies Applied Research Core Competencies, Cont’d

4. Telcordia Technologies Proprietary - Copyright 2003.  Wireless Networking –3 rd Generation (3G) –RF Technology –Digital Signal Processing for Wireless Comms –Wireless Architecture and Middleware –Wireless Applications/Support (WAP)  Optical Networking –Dense Wavelength Division Multiplexing (DWDM) –Routing in All-Optical Networks –Optical Network Engineering and Management –Quantum Computing and Quantum Cryptography Telcordia Technologies Applied Research Core Competencies, Cont’d

5. Telcordia Technologies Proprietary - Copyright 2003.  E-Business and Information Assurance –E-Commerce for Mobile Users (M-Commerce) –Data Mining and Information Extraction/Integration –Virtual Private Networks –Network Security  Plus Specialized Expertise In: –Speech Technology and Applications –Mathematical Sciences/Statistics (Algorithms, Network Traffic Modeling, Cryptography, etc.) –Cable Interconnection Technology Telcordia Technologies Applied Research Core Competencies, Cont’d

6. Telcordia Technologies Proprietary - Copyright 2003. Telcordia Technologies Applied Research Broadband Networking Group David Waring, Kenneth Kerpez, Thomas Banwell, Stefano Galli   Various aspects of DSL – –Standardization efforts; – –Loop and crosstalk identification; – –Crosstalk modeling; – –Dynamic Spectrum Management.   Power Line Communications   Home Networking (wired/wireless)

7. Telcordia Technologies Proprietary - Copyright 2003. Summary of presentation 1) 1)Hard and Soft Decisions in Adaptive Equalization 2)Linear and Nonlinear MMSE Channel Estimation 3)The Proposed Approach to Soft Detection 4) 4)Practical Applications 5) 5)Conclusions

8. Telcordia Technologies Proprietary - Copyright 2003. Hard decisions (Hard-Statistics) are extensively used communications, e.g. in adaptive receivers for channel estimation and tracking, in the feedback section of a DFE, etc. Hard-Decisions don’t give us an index of the reliability of the decisions but are the best we can do, if the decisions are correct. If the decisions are wrong, the use of Hard-Decisions may cause severe performance degradation (e.g. channel tracking loss, catastrophic error propagation, etc.). Hard-Statistics and Soft-Statistics

9. Telcordia Technologies Proprietary - Copyright 2003. Soft-Decisions (Soft-Statistics) are decisions on a transmitted symbol which also contain an index of the reliability of the decision. In general, Soft-Decisions contain all the information contained in Hard-Decisions, whereas the viceversa is not true. Hard-Statistics and Soft-Statistics (cont.) Recently, Soft-Decisions have been proven to be a useful tool in the following areas: Channel estimation and tracking; Combined adaptive equalization and decoding; Blind equalization based on second order statistics; Enhanced DFE with soft feedback section; Iterative decoding of parallel/serial concatenated coded streams.

10. Telcordia Technologies Proprietary - Copyright 2003. Essentially three kinds of soft-information: Nonlinear function applied to an estimate of the symbol. A Posteriori Probabilities (APPs); Estimate (usually, MMSE) of transmitted symbols; Is there any formal justification for their use? Are they related in any way? Hard-Statistics and Soft-Statistics (cont.)

11. Telcordia Technologies Proprietary - Copyright 2003. The Considered Communications System Time Division Multiple Access Transmission of independent time slots with preamble or midamble The System Model

12. Telcordia Technologies Proprietary - Copyright 2003. The System Model (cont.) Model of the observations (channel as linear filter, convolution): where:   (vector of the T S -sampled input delay-spread function g(t;  )); (ISI channel state vector); (complex zero-mean Gaussian noise sequence);

13. Telcordia Technologies Proprietary - Copyright 2003. Adaptive Receivers Based on Hard-Statistics Goal of an adaptive receiver Recover the transmitted data stream after the distortion caused by the channel, by continuously adapting to the time-varying channel. Basic components of an adaptive receiver Adaptive channel estimator: estimates on the basis of a certain criterion (LMS, MMSE, etc.) the distortion introduced by the channel and provides this estimate to the symbol/sequence detector. Symbol/sequence detector (equalizer): using the estimation of the channel, decides on the basis of a certain criterion (MAP, ML, ZF, etc.) what symbols were transmitted and then feeds them to the channel estimator.

14. Telcordia Technologies Proprietary - Copyright 2003. Adaptive Receivers Based on Hard-Statistics: major problems Decisions on symbols are best made (minimization of probability of error) if the detector “waits” and gathers more information from the received signal before deciding on the transmitted symbols. The amount of “waiting” is the decision delay. Example: Viterbi algorithm optimal if decision delay is infinite, but in practice a delay of five times the memory of the channel L is sufficient. Channel estimators process both the received signal and the decisions of the symbol/sequence detector to estimate the channel and output the estimate of the channel at the time the decisions were made. If the decision delay is too large, channel estimate is old. If decision delay is too small, decisions are not reliable.

15. Telcordia Technologies Proprietary - Copyright 2003. MLS detection and decision driven channel estimator   Low-delay tentative hard-decisions (d  (L-1)) for channel estimation.   Final hard-decisions at large delay (D  5(L-1)).   Trade-off between decision delay and prediction order of the channel estimate. Adaptive Receivers Based on Hard-Statistics: examples

16. Telcordia Technologies Proprietary - Copyright 2003. Effects of predicted channel estimates on system performance (BPSK over Rayleigh channel with Land Mobile fading spectrum) Adaptive Receivers Based on Hard-Statistics: examples

17. Telcordia Technologies Proprietary - Copyright 2003. Receivers based on the Per Survivor Processing (PSP) principle Kubo, Murakami, Fujino (1994) and Polydoros, Raheli (1995) There are S (L-1) distinct estimators for the S (L-1) states, respectively; Higher tracking capability at the expense of a much larger implementation complexity. Adaptive Receivers Based on Hard-Statistics: examples

18. Telcordia Technologies Proprietary - Copyright 2003. Channel Model Channel Model: Observation Observation : Goal MMSE Goal :, i.e. MMSE estimation of the channel impulse response The Linear MMSE recursive estimate of the channel impulse response: MMSE Channel Estimation: Linear or Nonlinear?

19. Telcordia Technologies Proprietary - Copyright 2003. The one-step MMSE prediction of the observation correct Common semplification: assume correct hard decisions. MMSE Channel Estimation: Linear or Nonlinear? pdf of G(i) conditioned to y(i) and x(i) is Gaussian  Linear MMSE estimation (Kalman) of G(i) is optimal

20. Telcordia Technologies Proprietary - Copyright 2003. However, the problem is intrinsecally non-Gaussian: pdf of x(i) or G(i) conditioned to y(i) is not Gaussian  Optimal MMSE channel estimator is nonlinear MMSE Channel Estimation: Linear or Nonlinear? nonlinear Goal:, nonlinear MMSE estimation of the channel impulse response

21. Telcordia Technologies Proprietary - Copyright 2003. In estimation theory, it is useful to define the following sequences: In fact, the following sequences: (estimate innovation sequence) (observation innovation sequence) have special properties depending on the nature of the estimator: Linear estimation : uncorrelated sequences. Nonlinear estimation : Martingale Difference (MD) sequences. Nonlinear MMSE Channel Estimation

22. Telcordia Technologies Proprietary - Copyright 2003. Nonlinear MMSE Channel Estimation (cont.)

23. Telcordia Technologies Proprietary - Copyright 2003. adaptedmeasurable Given a probability space ( , F, P ), the sequence {C} is said to be adapted to { B }, if C(i/i) is measurable with respect to B i for any i, where { B }={ B i, i=1, 2,...} is an increasing sequence of  -algebras of subsets in F. not recursively computable Since B i can be viewed as containing the past of all sequences of interest up to instant i, the property of being an adapted sequence implies that the filtering gain C(i/i) is not recursively computable. Nonlinear MMSE Channel Estimation (cont.)

24. Telcordia Technologies Proprietary - Copyright 2003. predictable measurable In order to have a recursive estimator, it would be necessary that the gain sequence C(i/i) be predictable not adapted or, equivalently, that C(i/i) is measurable with respect to B i-1 for any i. Therefore: false Unfortunately, the MD-Representation theorem has been proven to be false in the predictable form for the case of discrete-time observations in white Gaussian noise!!! Nonlinear MMSE Channel Estimation (cont.)

25. Telcordia Technologies Proprietary - Copyright 2003. Non Linear MMSE estimation: E. Baccarelli, R. Cusani, S. Galli, “A Novel Adaptive Receiver with Enhanced Tracking Capability for TDMA-Based Mobile Radio Communications”, IEEE JSAC-Special Issue on Wireless Communications (Part II), Vol. 16, No. 8, Dec. 1998. Formal approach via martingales is not trivial; The exact solution does not exist in the recursive and finite dimensional form; Structure of the estimator is fixed: Kalman-like; Computational complexity issues; Is there another way? Nonlinear MMSE Channel Estimation (cont.)

26. Telcordia Technologies Proprietary - Copyright 2003. Let us consider again the linear MMSE recursive estimate of the channel impulse response: Conjecture: this approximation is less harsh than assuming correct hard decisions. Soft-Decision Based Channel Estimation

27. Telcordia Technologies Proprietary - Copyright 2003.  NL-MMSE prediction of the states of the ISI channel or NL-MMSE filtered and fixed-lag smoothed estimates of the transmitted symbols Soft-Decision Based Channel Estimation (cont.)

28. Telcordia Technologies Proprietary - Copyright 2003. Goal: Non-linear MMSE estimation of transmitted symbol s(i) NL-MMSE estimation of the channel impulse response  NL-MMSE estimate of the ISI channel state vector Soft-Decision Based Channel Estimation (cont.)

29. Telcordia Technologies Proprietary - Copyright 2003. NL- MMSE estimation of transmitted symbols Tarköy (ISIT ’95), Wang & Poor (IEEE Trans. Comm. ’99) Only filtered estimates may be obtained. Therefore, it is optimal only for channels with no memory. Soft-Decision Based Channel Estimation (cont.)

30. Telcordia Technologies Proprietary - Copyright 2003. NL- MMSE estimation of transmitted symbols (ISIT ’00; IEEE Trans. on Comm., Dec. 2002) New approach New approach: where 0  D  L-1 Both filtered and fixed-lag smoothed estimates may be obtained. Therefore, it is optimal also for channels with memory. Soft-Decision Based Channel Estimation (cont.)

31. Telcordia Technologies Proprietary - Copyright 2003. Optimal NL-MMSE symbol estimation as a linear transformation of the APP vector of the channel state. Soft-Decision Based Channel Estimation (cont.)

32. Telcordia Technologies Proprietary - Copyright 2003.   The channel estimator is fed by the more informative NL-MMSE estimates of the transmitted symbols and not by the usual hard-decided data.   The APPs are recursively computed and delivered to the channel estimator with no delay. Practical applications: adaptive equalization

33. Telcordia Technologies Proprietary - Copyright 2003.   A MLS equalizer can operate efficiently at a high decision delay, e.g. at a decision delay equal to the length of the TDMA-slot.   The VA can build the trellis with more reliable zero-delayed channel estimates and in parallel with channel tracking. Practical application: adaptive equalization (cont.)

34. Telcordia Technologies Proprietary - Copyright 2003. Six equal-powered taps with land mobile fading spectrum (BPSK modulation - L p =12, L f =60 - B D T S = 10 -4 ) Practical application: adaptive equalization (cont.)

35. Telcordia Technologies Proprietary - Copyright 2003. Six equal-powered taps with land mobile fading spectrum (BPSK modulation - L p = 20, L f = 100 - B D T S = 10 -3 ) Practical application: adaptive equalization (cont.)

36. Telcordia Technologies Proprietary - Copyright 2003. HF link: CCIR Moderate conditions (  = 1 ms;B D =0.5 Hz) (QPSK modulation - L p =15, L f =50 - B D T S = 4.17·10 -4 ) Practical application: adaptive equalization (cont.)

37. Telcordia Technologies Proprietary - Copyright 2003. Practical application: enhanced DFE Digital Feedback Equalization (DFE) Pros: optimum (MMSE sense), cheap, adaptive channel estimation. Cons: risk of catastrophic error propagation. MAP or MLSE are better than the ideal DFE (at least 3 dB) If channel impulse response is short: MAP or MLSE are the preferred choice. If channel impulse response is long: DFE is the only practical approach unless reduced state techniques are employed for the MAP or MLSE solutions

38. Telcordia Technologies Proprietary - Copyright 2003. Practical application: enhanced DFE (cont.) The soft approach (Globecom 1999) Reduced state MAP or MLSE receivers with feedback filter to shorten the long impulse response; Relax the overly optimistic assumption of error free decisions; non-linear MMSE Do not use hard decisions in the feedback section but the non-linear MMSE estimates of the transmitted symbols; A Posteriori Probabilities The non-linear MMSE estimates are computed on the basis of the A Posteriori Probabilities (APP, soft statistics) delivered at small delay by a MAP receiver;

39. Telcordia Technologies Proprietary - Copyright 2003. Fig. 5 Practical application: enhanced DFE (cont.)

40. Telcordia Technologies Proprietary - Copyright 2003. Practical application: multiuser detection Successive Cancellation MUD Orders the users from the strongest to the weakest: P 1 >P 2 > … >P K ; Detect the data sequence of the first (strongest) user; Subtract the decoded stream (hard decisions) from the observations; Detect the data sequence of the second user and so on;

41. Telcordia Technologies Proprietary - Copyright 2003. Practical application: multiuser detection (cont.) The soft approach (DARPA MIMO project, 2002) non-linear MMSE Do not use hard decisions in the subtraction of the decoded users but the non-linear MMSE estimates of the transmitted symbols; Better performances are obtained if the fixed-lag smoothed estimates are used in place of the filtered ones; w0w0 Detector hk0hk0 Hard-decisions Conventional Hard SIC w0w0 APP Computer hk0hk0 Linear Transformation NL-MMSE Fixed-lag Smoothed Estimates of the Transmitted Symbol Proposed Soft SIC

42. Telcordia Technologies Proprietary - Copyright 2003. Sufficient condition for channel identification: Probability distribution of must be equal to the probability distribution of {s(i)}. This condition implies that the following cost function must be minimized:, with p>2 Sato (‘75), Godard (‘80), Benveniste-Goursat (‘80), Picchi-Prati (‘80), Shalvi-Weistein (‘90) Main disadvantages: Slow convergence: several thousands of observation samples are needed to achieve channel identification; The channel is estimated with a high residual MSE due to the use of nonconvex cost functions. The channel estimate is not always available Practical Applications: Blind Equalization

43. Telcordia Technologies Proprietary - Copyright 2003. The channel-estimator is fed with the soft information given by the APPs of the channel Practical Applications: Blind Equalization (cont.) IEEE Trans. on Signal Processing, July 2001

44. Telcordia Technologies Proprietary - Copyright 2003. Channel: g=[1, 0, -1] Practical Applications: Blind Equalization (cont.)

45. Telcordia Technologies Proprietary - Copyright 2003. As the transmitted bit-rate increases, ISI increases and powerful channel estimators become more necessary. As the transmitted bit-rate increases, ISI increases and powerful channel estimators become more necessary. Hard-decision driven channel estimators are sub-optimal. Hard-decision driven channel estimators are sub-optimal. Optimal channel estimation implies the computation of the non- linear filtered and fixed-lag smoothed estimates of the transmitted symbols. Optimal channel estimation implies the computation of the non- linear filtered and fixed-lag smoothed estimates of the transmitted symbols. A new and simpler method for generating NL-MMSE filtered and fixed-lag smoothed estimates of the transmitted symbols via APPs has been proposed. A new and simpler method for generating NL-MMSE filtered and fixed-lag smoothed estimates of the transmitted symbols via APPs has been proposed. Conclusions

46. Telcordia Technologies Proprietary - Copyright 2003. Conclusions NL-MMSE filtered and fixed-lag smoothed estimates of the transmitted symbols can be seen as optimal soft information, and their use is a consequence of the correct statement of the problem of MMSE estimation. NL-MMSE filtered and fixed-lag smoothed estimates of the transmitted symbols can be seen as optimal soft information, and their use is a consequence of the correct statement of the problem of MMSE estimation. The proposed method makes SbS-MAP receivers very appealing.The proposed method makes SbS-MAP receivers very appealing. The proposed approach can be applied to all those problems that admit a space-state representation. The proposed approach can be applied to all those problems that admit a space-state representation. Several fields of application, especially all those situations where hard decisions are employed despite their low reliability. Several fields of application, especially all those situations where hard decisions are employed despite their low reliability.