1. BMI II SS06 – Class 2 “Linear Systems 2” Slide 1 Biomedical Imaging II Class 2 – Mathematical Preliminaries: Signal Transfer and Linear Systems Theory 2/6/06

2. BMI II SS06 – Class 2 “Linear Systems 2” Slide 2 Class objectives Topics you should be familiar with after lecture: Impulse response function (irf) of a linear system (LS) Convolutions Fourier transform (FT)

3. BMI II SS06 – Class 2 “Linear Systems 2” Slide 3 Overview of topic Goal: To describe a physical system with a mathematical model Example : medical imaging Energy sourcedetector(s)human body "system" quantum image energy conversion analog / digital image detection, storage "system" imaging algorithm ………

4. BMI II SS06 – Class 2 “Linear Systems 2” Slide 4 Signal transfer by a physical system Signal transferred by a system System input is a function h(x) System operates on input function (system can be described by a mathematical operator) System output is function S{h(x)} Objective: To come up with operators that accurately model systems of interest S h(x)h(x)S{h(x)} InputSystemOutput

5. BMI II SS06 – Class 2 “Linear Systems 2” Slide 5 Linear systems Additivity Homogeneity Preceding two can be combined into a single property, which is actually a definition of linearity:

6. BMI II SS06 – Class 2 “Linear Systems 2” Slide 6 LS significance and validity Why is it desirable to deal with LS? Decomposition (analysis) and superposition (synthesis) of signals System acts individually on signal components No signal “mixing” Simplifies qualitative and quantitative measurements Lets us separate S{h(x)} into two independent factors: the source, or driving, term, and the system’s impulse response function (irf)

7. BMI II SS06 – Class 2 “Linear Systems 2” Slide 7 Rectangular input function (Rectangular pulse) S h(x)h(x)S{h(x)} InputSystemOutput h(x), h(t) x, t 0

8. BMI II SS06 – Class 2 “Linear Systems 2” Slide 8 Important rectangular pulse special cases H(x-x 0 ), H(t-t 0 ) x, t 0 Step function, Heaviside function (1) x 0, t 0 u(x), u(t) x, t 0 x 1, t 1 x 2, t 2 (x 2 -x 1 ) -1, (t 2 -t 1 ) -1 Unit rectangular (or square) pulse

9. BMI II SS06 – Class 2 “Linear Systems 2” Slide 9 Limiting case of unit pulse x, t 0 As the pulse narrows we also make it higher, such that the area under the pulse is constant. (Variable power, constant energy.) We can imagine making the pulse steadily narrower (briefer) until it has zero width but still has unit area! A pulse of that type (zero width, unit area) is called an impulse.

10. BMI II SS06 – Class 2 “Linear Systems 2” Slide 10 Impulse response function (irf) S h(x)h(x)S{h(x)} InputSystemOutput impulse function goes in…impulse response function (irf) comes out! Note: irf has finite duration. Any input function whose width/duration is << that of the irf is effectively an impulse with respect to that system. But the same input might not be an impulse wrt a different system.

11. BMI II SS06 – Class 2 “Linear Systems 2” Slide 11 Properties of impulse function Mathematical description of impulse function by delta (or Dirac) function: Definition: Sifting property:

12. BMI II SS06 – Class 2 “Linear Systems 2” Slide 12 Properties of impulse function Mathematical description of impulse function by delta (or Dirac) function: Definition: Sifting property: Equivalently if a < b, then: Note that no true function can have the properties given in the definition of . It is more proper to call  a functional or a generalized function.

13. BMI II SS06 – Class 2 “Linear Systems 2” Slide 13 Relation between impulse and step functions H(x-x 0 ), H(t-t 0 ) x, t 0 Step function, Heaviside function (1) x 0, t 0

14. BMI II SS06 – Class 2 “Linear Systems 2” Slide 14 Filter function = irf of the system System (= filter) response: Square pulse smearing out an impulse function evenly over N time points: N = 5 05-5 t 0.2 0.4 0.0 0.6 0.8 1.0 s System (Filter) 0510-5 t 0.2 0.4 0.0 0.6 0.8 1.0 s

15. BMI II SS06 – Class 2 “Linear Systems 2” Slide 15 Output for arbitrary input signals is given by superposition principle (Linearity!) Think of an arbitrary input function as a sequence of impulse functions, of varying strengths (areas), tightly packed together Then the defining property of an LS, (S{a·h 1 + b·h 2 } = a·S{h 1 } + b·S{h 2 }) tells us that the overall system response is the sum of the corresponding irfs, properly scaled and shifted. Significance of the irf of a LS =

16. BMI II SS06 – Class 2 “Linear Systems 2” Slide 16 Significance of the irf of a LS To state the same idea mathematically, LS output given by convolution of input signal and irf: Notice that the sum of these two arguments is a constant or t

17. BMI II SS06 – Class 2 “Linear Systems 2” Slide 17 1/6 1 2 3 4 5 6 1/6 2 3 4 5 6 7 8 9 10 11 12 Real-world convolution example

18. BMI II SS06 – Class 2 “Linear Systems 2” Slide 18 Convolution Example Moving – average filter to smooth noisy data Corresponds to convolution of signal with a rectangular pulse of height 1/N and length N t s 1/5[s(1) + s(2) + s(3) + s(4) + s(5)] 1/5[s(2) + s(3) + s(4) + s(5) + s(6)] 1/5[s(7) + s(8) + s(9) + s(10) + s(11)]

19. BMI II SS06 – Class 2 “Linear Systems 2” Slide 19 Signal with added noise

20. BMI II SS06 – Class 2 “Linear Systems 2” Slide 20 Result of smoothing

21. BMI II SS06 – Class 2 “Linear Systems 2” Slide 21 Analytic convolution example I

22. BMI II SS06 – Class 2 “Linear Systems 2” Slide 22 Analytic convolution example II

23. BMI II SS06 – Class 2 “Linear Systems 2” Slide 23 Analytic convolution example III

24. BMI II SS06 – Class 2 “Linear Systems 2” Slide 24 Analytic convolution example III

25. BMI II SS06 – Class 2 “Linear Systems 2” Slide 25 Analytic convolution example III

26. BMI II SS06 – Class 2 “Linear Systems 2” Slide 26 f(t)*g(t)f(t)*g(t) Analytic convolution example IV

27. BMI II SS06 – Class 2 “Linear Systems 2” Slide 27 Analytic convolution example I

28. BMI II SS06 – Class 2 “Linear Systems 2” Slide 28 Mathematical footnote Imaginary numbers: Any number ki, where k is real, is an imaginary number Complex numbers: any number of the form a + bi, where a and b are real numbers Each complex number has an equivalent representation as a complex exponential (same thing as converting from rectangular to polar coordinates): Euler’s theorem: e ix = cosx + isinx.

29. BMI II SS06 – Class 2 “Linear Systems 2” Slide 29 System response to a sinusoidal input Frequency domain description: sinusoids as input functions Evaluation by convolution: Sinusoids are eigenfunctions of LS (i.e. are multiplied by complex, frequency dependent factor  amplitude scaling, phase shift) Convolution theorem gives:

30. BMI II SS06 – Class 2 “Linear Systems 2” Slide 30 Excursion: Fourier Transform (FT) The one-dimensional Fourier Transform is given by F (  ) is in general a complex number, with real and imaginary parts Mostly interested in magnitude or power ( = M (  ) 2 = | F (  )| 2 ) of FT (the spectrum of f(t) )

31. BMI II SS06 – Class 2 “Linear Systems 2” Slide 31 Inverse FT The one-dimensional inverse Fourier Transform (IFT) is given by The Fourier transform and its inverse can be interpreted as a mathematical technique for converting time–domain data to frequency–domain data, and vice versa.

32. BMI II SS06 – Class 2 “Linear Systems 2” Slide 32 Detailed FT example 1 t f(t)f(t) t=at=a t=bt=b K 0 Fourier transform ?

33. BMI II SS06 – Class 2 “Linear Systems 2” Slide 33 a = 0.9, b = 1.1, K = 2 a = 0.4, b = 0.6, K = 2 M M M(F)M(F) M(F)M(F) ω ω Detailed FT example 1

34. BMI II SS06 – Class 2 “Linear Systems 2” Slide 34 Instructive special cases A) K = 1, a = 0, b = 1: M M ω

35. BMI II SS06 – Class 2 “Linear Systems 2” Slide 35 Result of smoothing

36. BMI II SS06 – Class 2 “Linear Systems 2” Slide 36 Result of smoothing (freq. domain) I Original signal

37. BMI II SS06 – Class 2 “Linear Systems 2” Slide 37 Original signal vs. filtered signal Result of smoothing (freq. domain) II

38. BMI II SS06 – Class 2 “Linear Systems 2” Slide 38 Properties of Fourier Transform Linearity ö[af(x) + bg(x)] = aö[f(x)] + bö[g(x)] Scaling Shifting ö[f(x-a)] = ö[f(x)]e -iau Convolution theorem ö[f(x)g(x)] = ö[f(x)]  ö[g(x)], ö[f(x)  g(x)] = ö[f(x)]ö[g(x)] Parseval’s theorem

39. BMI II SS06 – Class 2 “Linear Systems 2” Slide 39 Parseval’s Theorem The total energy in the time domain is the same as the total energy in the frequency domain. (area under curves is the same)

40. BMI II SS06 – Class 2 “Linear Systems 2” Slide 40 FT Energy or Power H2OH2O Wave generation tank Each H 2 O molecule at the water surface travels up and down a distance of 4A during time T. Its average speed during this time is v avg = 4A/T. Wave amplitude = A. Wave period = T. Let’s increase the force with which we strike the water until the wave amplitude doubles. By what factor does its average speed change? What about its average kinetic energy?

41. BMI II SS06 – Class 2 “Linear Systems 2” Slide 41 Class objectives Topics you should be familiar with after lecture: FT of an impulse (delta) function FT of a sequence of impulses Equivalence of a discrete (or sampled) function to the product of a continuous function and a “comb” of impulses Relationship between FT of a discrete function and the continuous function from which it is obtained Formulas for discrete Fourier transform (DFT) Essential properties of DFT, and differences between DFT and continuous FT What aliasing is What a band-limited function is Definition of correlation Relationships/differences between correlation and convolution

42. BMI II SS06 – Class 2 “Linear Systems 2” Slide 42 Impulse function FT

43. BMI II SS06 – Class 2 “Linear Systems 2” Slide 43 FT of multiple impulse functions But what does an FT of this sort look like? Comb function t 0 = 0 t 0 = 1 t 0 = -1 t 0 = 2 t 0 = -2

44. BMI II SS06 – Class 2 “Linear Systems 2” Slide 44 Partial sums of comb function FT

45. BMI II SS06 – Class 2 “Linear Systems 2” Slide 45 Comb function FT Time Domain: Comb of  -functions separated by T. Frequency Domain: Comb of  -functions with separation 1/T. f

46. BMI II SS06 – Class 2 “Linear Systems 2” Slide 46 Analytic, continuous-function FT example FT

47. BMI II SS06 – Class 2 “Linear Systems 2” Slide 47 f * Discrete-function FT example FT

48. BMI II SS06 – Class 2 “Linear Systems 2” Slide 48 Convolution of comb and non-periodic FT

49. BMI II SS06 – Class 2 “Linear Systems 2” Slide 49 Discrete Fourier Transform (DFT) In real life, data are typically sampled at N evenly spaced time intervals Δt. To take Fourier Transform of discretely sampled signals, we need the DFT of a sequence {u(n), n = 0, 1,...., N-1} The inverse discrete Fourier Transform is given by

50. BMI II SS06 – Class 2 “Linear Systems 2” Slide 50 DFT – What happens when k = 0 or N/2 ?

51. BMI II SS06 – Class 2 “Linear Systems 2” Slide 51 Discrete Fourier Transform – Periodicity I

52. BMI II SS06 – Class 2 “Linear Systems 2” Slide 52 Discrete Fourier Transform – Periodicity II But 2mk is an even integer, and therefore e 2  imk = 1 Accordingly, e 2  ink/N is periodic with period N

53. BMI II SS06 – Class 2 “Linear Systems 2” Slide 53 Discrete functions and aliasing I

54. BMI II SS06 – Class 2 “Linear Systems 2” Slide 54 Discrete functions and aliasing II

55. BMI II SS06 – Class 2 “Linear Systems 2” Slide 55 Discrete functions and aliasing III

56. BMI II SS06 – Class 2 “Linear Systems 2” Slide 56 Discrete functions and aliasing IV

57. BMI II SS06 – Class 2 “Linear Systems 2” Slide 57 Discrete functions and aliasing V When the sampling interval is Δt, the highest frequency that can be accurately reproduced by the samples is f t = 1/(2Δt) (or ω t =  /Δt). (Nyquist) When the sampling interval is Δt, the limiting frequency corresponds to exactly two samples per wavelength or period. Note, however, that some loss of information occurs at the Nyquist frequency. We get a real value in the DFT, while the continuous function being sampled might well have had a non–zero imaginary part at that frequency. Another way of saying the same thing: the DFT gets the amplitude right at the Nyquist frequency, but loses the phase.

58. BMI II SS06 – Class 2 “Linear Systems 2” Slide 58 Discrete functions and aliasing VI What can we do to get the correct phase information at (or above) ω c  ω Nyquist ? Suggestion: Increase the number of samples, but how? (Smaller Δ t? Larger number of samples? Both?)

59. BMI II SS06 – Class 2 “Linear Systems 2” Slide 59 Discrete functions and aliasing VII When a continuous function is sampled at rate Δt, the DFT contains all frequencies above ω c aliased down to apparent frequencies less than ω c. The only good (I said good, not easy) preventives for aliasing are: 1. increase sampling rate until ω c reaches a value above which it is known a priori that the continuous signal contains negligible spectral energy; 2. have the continuous signal pass through a low–pass filter, with a frequency cutoff less than ω c, before the signal is sampled. If either of the above is successfully carried out, then the continuous signal is said to be band–limited.

60. BMI II SS06 – Class 2 “Linear Systems 2” Slide 60 Modulation Transfer Function (MTF) FT of irf / psf is called the system transfer function T(  ) / T(u,v) T yields the (spatial) frequency components transmitted by the linear system (amplitude and phase) The normalized expression T(u,v) / T(0,0) is the optical transfer function O(u,v) The magnitude of T can be thought of as a filter function The magnitude of the normalized function O(u,v) is called the modulation transfer function MTF used to describe resolution frequency-depending contrast frequency-depending sensitivity

61. BMI II SS06 – Class 2 “Linear Systems 2” Slide 61 Convolution vs. Correlation Sum is a constant Difference is a constant Correlation is a measure of the similarity between f and g. t is a variable to account for the possibility that it might look as though f and g are very different, but it turns out that g is simply displaced in time relative to f.

62. BMI II SS06 – Class 2 “Linear Systems 2” Slide 62 Correlation example

63. BMI II SS06 – Class 2 “Linear Systems 2” Slide 63 Convolution vs. correlation: frequency domain