1. Analysis of distorted waveforms using parametric spectrum estimation methods and robust averaging Zbigniew LEONOWICZ 13 th Workshop on High Voltage Engineering Söllerhaus Austria 11-15.09.2006

2. Robust averaging Averaging is probably the most widely used basic statistical procedure in experimental science.Averaging is probably the most widely used basic statistical procedure in experimental science. Estimation of the location of data („central tendency”) in the presence of random variations among the observationsEstimation of the location of data („central tendency”) in the presence of random variations among the observations Data variations can be a result of variations in the phenomenon of interest or of some unavoidable measuring errors.Data variations can be a result of variations in the phenomenon of interest or of some unavoidable measuring errors. In signal processing terms, this can be considered as contamination of useful „signal” by useless „noise” linearly added to it.In signal processing terms, this can be considered as contamination of useful „signal” by useless „noise” linearly added to it. Since the noise usually has zero mean, averaging minimizes its contribution, while the signal is preserved, and the signal to noise ratio is improvedSince the noise usually has zero mean, averaging minimizes its contribution, while the signal is preserved, and the signal to noise ratio is improved

3. Synchronization Averaging consists of applying of any statistical procedure to extract the useful information from the background noise.Averaging consists of applying of any statistical procedure to extract the useful information from the background noise. When useful data are time-locked to some event and the noise is not time-locked, it allows the cancellation of the noise by simple point-by- point data summation.When useful data are time-locked to some event and the noise is not time-locked, it allows the cancellation of the noise by simple point-by- point data summation. This procedure is equivalent to the use of the arithmetic meanThis procedure is equivalent to the use of the arithmetic mean

4. Review of robust avearging methods Sensitivity of an estimator to the presence of outliers (i.e. data points that deviate from the pattern set by the majority of the data set)Sensitivity of an estimator to the presence of outliers (i.e. data points that deviate from the pattern set by the majority of the data set) Robustness of an estimator is measured by the breakdown valueRobustness of an estimator is measured by the breakdown value How many data points need to be replaced by arbitrary values in order to make the estimator explode (tend to infinity) or implode (tend to zero) ?How many data points need to be replaced by arbitrary values in order to make the estimator explode (tend to infinity) or implode (tend to zero) ? Arithmetic mean has 0% breakdownArithmetic mean has 0% breakdown Median is very robust with breakdown value 50%Median is very robust with breakdown value 50%

5. Robust location estimators Many location estimators can be presented in unified way by ordering the values of the sample asMany location estimators can be presented in unified way by ordering the values of the sample as and then applying the weight function and then applying the weight function where is a function designed to reduce the influence of certain observations (data points) in form of weighting and represents ordered data.where is a function designed to reduce the influence of certain observations (data points) in form of weighting and represents ordered data.

6. Examples MedianMedian When the data have the size of (2M+1), the median is the value of the (M +1) th ordered observation. Trimmed meanTrimmed mean For the  -trimmed mean (where p =  N) the weights can be defined as: p highest and p lowest samples are removed.

7. Winsorized mean Winsorized mean replaces each observation in each  fraction (p =  N) of the tail of the distribution by the value of the nearest unaffected observation.Winsorized mean replaces each observation in each  fraction (p =  N) of the tail of the distribution by the value of the nearest unaffected observation. 0  p  0,25N usually, depending on the heaviness of the tails of the distribution. 0  p  0,25N usually, depending on the heaviness of the tails of the distribution.

8. Weight functions

9. Weight functions - other TL-mean applies higher weights for the middle observationsTL-mean applies higher weights for the middle observations tanh estimator applies smoothly changing weights to the values close to extreme, it can be set to ignore extreme valuestanh estimator applies smoothly changing weights to the values close to extreme, it can be set to ignore extreme values

10. Comparison

11. Investigations IEC harmonic and interharmonic subgroups calculation IEC Std 61000-4-7, 61000-4-30IEC harmonic and interharmonic subgroups calculation IEC Std 61000-4-7, 61000-4-30 DFT with 5 Hz resolution in frequency characterize the waveform distortionsDFT with 5 Hz resolution in frequency characterize the waveform distortions

12. Parametric methods MUSICMUSIC Eigenvalues of the correlation matrix which correspond to the noise subspace used for parameter estimation ESPRITESPRIT based on naturally existing shift invariance between the discrete time series, which leads to rotational invariance between the corresponding signal subspaces. Uses signal subspace.

13. Progr. average of harmonic groups dc arc furnace supplydc arc furnace supply 11th harmonic group11th harmonic group 2nd interharmonic group2nd interharmonic group

14. Results MSE Method MSE groups MSE subgroups DFT0.0590.791 ESPRIT0.0210.169 MUSIC0.0270.201

15. Advantage of Winsorized mean When comparing values of power quality indices obtained from different parts of the same recorded waveform, a high variability of results appears. To alleviate this problem, winsorized mean was appplied to compute averages from spectral data. When using the value of a=0.2 which means that 20% of ordered data points were discarded and replaced by nearest unaffected data.When comparing values of power quality indices obtained from different parts of the same recorded waveform, a high variability of results appears. To alleviate this problem, winsorized mean was appplied to compute averages from spectral data. When using the value of a=0.2 which means that 20% of ordered data points were discarded and replaced by nearest unaffected data. In such way the outliers were removed and replaced by data, which are assumed to belong to “true” spectral content of investigated waveform.In such way the outliers were removed and replaced by data, which are assumed to belong to “true” spectral content of investigated waveform. The use of winsorized mean instead of usual arithmetic mean allowed reducing the variance of results by nearly 35%.The use of winsorized mean instead of usual arithmetic mean allowed reducing the variance of results by nearly 35%.

16. Conclusions Results show that the highest improvement of accuracy can be obtained by using the ESPRIT method (especially for interharmonics estimation), closely followed by MUSIC method, which outperform classical DFT approach by over 50%.Results show that the highest improvement of accuracy can be obtained by using the ESPRIT method (especially for interharmonics estimation), closely followed by MUSIC method, which outperform classical DFT approach by over 50%. Partially stochastic nature of investigated arc furnace waveforms caused high variability of calculated power quality indices. The use of robust averaging (winsorized mean) helped to reduce this unwanted variability.Partially stochastic nature of investigated arc furnace waveforms caused high variability of calculated power quality indices. The use of robust averaging (winsorized mean) helped to reduce this unwanted variability.

17. Conclusions Trimmed estimators are a class of robust estimators of data locations which can help to improve averaging of experimental data when: number of experiments is small data are highly nonstationary data include outliers. Their advantages can be understood as a reasonable compromise between median which is very robust but discard too much information and arithmetic mean conventionally used for averaging which use all data but, due of this, is sensitive to outliers. Additional improvement of averaging can be gained by introducing advanced weighting of ordered data