1. Pre-processing of NIR
Åsmund Rinnan

2. Introduction NIR
Fructose
Glucose 25 50 75 75 50 25

3. Introduction NIR Baseline with a slope/ curve Nonlinearity
Fructose
Glucose 25 50 75 75 50 25
A “fat” baseline

4. Introduction Techniques
Reference dependent
O-PLS
OSC OS
SIS
Reference independent
MSC/ ISC
EMSC/ EISC
SNV
Detrend
Normalization
Savitzky-Golay
Norris-Williams
Finite difference

5. Introduction Techniques
Correction of light scatter
MSC
EMSC
Detrend
SNV
Derivation
Finite difference
Savitzky-Golay

6. Effect of pre-processing Before
Specular effect

7. Effect of pre-processing After

8. MSC
Raw
Reference

9. MSC
Raw
Reference

10. MSC Raw spectrum b = Slope a = Intercept Reference
P Geladi, D MacDougal, H Martens (1985): Linearization and scatter correction for near-infrared reflectance spectra of meat, Applied Spectroscopy, 39,

11. MSC

12. Extended MSC Wavelength Slope & Intercept Known spectra correction = =
Basic MSC =
Detrend
H. Martens, E. Stark (1991): Extended multiplicative signal correction and spectral interference subtraction: new preprocessing methods for near infrared spectroscopy, Journal of Pharmaceutical and Biomedicinal Analysis, 9,

13. Extended MSC The correction Calculated Step 1: Step 2:
H. Martens, E. Stark (1991): Extended multiplicative signal correction and spectral interference subtraction: new preprocessing methods for near infrared spectroscopy, Journal of Pharmaceutical and Biomedicinal Analysis, 9,

14. X X Extended MSC Summary Parameter setting Paraemter setting MSC
Detrend

15. SNV
R.J. Barnes, M.S. Dhanoa, S.J. Lister (1989): Standard Normal Variate Transformation and De-trending of Near-Infrared Diffuse Reflectance Spectra, Applied Spectroscopy, 43,

16. SNV vs MSC
MSC
SNV

17. SNV vs. MSC Classificafication of barley

18. SNV vs. MSC SNV MSC Normalization Reference needed No reference
Outlier sensitive
Only similar spectra
No normalization

19. Derivation

20. Derivation Finite difference

21. Savitzky-Golay vs. Finite difference
Derivative
Method 1 2 3 4
1st SG
0.38 (5)
0.40 (5)
0.47 (6)
0.46 (3)
Finite difference
0.34 (5)
0.58 (4)
0.73 (4)
2nd
0.34 (6)
0.51 (4)
0.67 (5)
0.72 (4)
0.88 (5)
0.91 (5)
0.42 (6)

22. Derivation Savitzky-Golay
A Savitsky, M J E Golay (1964): Smoothing and differentiation of data by simplified least squares procedures, Analytical Chemistry, 36 (8),

23. Derivation Savitzky-Golay
Smoothing 11 9 7 5 3
A Savitsky, M J E Golay (1964): Smoothing and differentiation of data by simplified least squares procedures, Analytical Chemistry, 36 (8),

24. Derivation Savitzky-Golay
Smoothing
Polynomial 7 4 2 5 1 3
A Savitsky, M J E Golay (1964): Smoothing and differentiation of data by simplified least squares procedures, Analytical Chemistry, 36 (8),

25. Derivation Savitzky-Golay
7 point smoothing
15 point smoothing
2nd order derivative

26. Summary Pre-processing of NIR
Run the following for near to optimal results
MSC with
Only 1st order reference correction or SNV
1st order reference correction and 2nd order wavelength correction
Only 2nd order wavelength correction
Savitzky-Golay derivation with
2nd order polynomial smoothing
7 points smoothing for the 1st derivative
9 points smoothing for the 2nd derivative

27. Compression factor Barley data
Pre-processing MB MH
Raw
0.0 %
MSC1
88.0 %
90.4 %
MSC2
92.3 %
94.4 %
SNV
SNV + Detrend
90.9 %
93.9 %
SG1
91.6 %
92.8 %
SG2
93.3 %