1. Predictive Analytics derived from HVAC and PMU data at UCSD Chuck Wells Industry Principal OSIsoft, LLC 1

2. Outline What is big data Structured versus unstructured data sets Tools used for developing analytics Feature extraction methods Focus on PMU data analytics 2

3. Big Data Typically large volumes of structured or unstructured data. Example of the HVAC data set at UCSD – 2 years of one minute interval data from 106,000 sensors – Over 530 Gbytes of structured text – Each day is one file of 1440 rows, 106,000 columns 3

4. PMU data sets An Example: Planned Indonesia Wide Area Measurement System (WAMS) PMU based measurement system Twenty regions – Each with 40 PMUs Each PMU has 40 measurement – Sampling rate is 50 Hz – Each PMU generates 2000 samples per second 4

5. PMU data archives Raw data (1.416 Gbytes per day) Calculated angle differences (91.88 GB/d) Computed FFTs (7250.98 GB/day) ~7.2 PB/d 5

6. Moving Window FFTs Use FFT spectra to extract features from the data sets Typically use 1024 wide windows running at 50 or 60 Hz rate to compute spectrum 6

7. Examples 7

8. PMU Frequency Data

9. Sampling PMU Frequency Data and Fast Fourier Transformation (FFT) Transforming Frequency Data to FFT Data – 23 samples of Frequency Data was taken from the PMU at different times – The FFT was computed for each sample – Each FFT was standardized by setting the max value to 1 – The following slides are the standardized FFT for the various time samples

10. X-Axis = FrequencyY-Axis: Magnitude FFT at Various Time (1 of 4)

11. FFT at Various Time (2 of 4) X-Axis = FrequencyY-Axis: Magnitude

12. FFT at Various Time (3 of 4) X-Axis = FrequencyY-Axis: Magnitude

13. FFT at Various Time (4 of 4) X-Axis = FrequencyY-Axis: Magnitude

14. Time Series Representation and Similarity Measure Transforming FFT Data into FFT Bins – For each preceding sample, FFT Frequencies are discretized into 25 bins – For each bin the mean and the sum are calculated – Correlation matrix comparing the corresponding event and control frequency bins

15. FFT Correlation Matrix Control GroupEvent Group

16. Simple Anomaly Detection Benford’s Law – Also called the First-Digit Law, refers to the frequency distribution of digits in many (but not all) real-life sources of data. In this distribution, the number 1 occurs as the leading digit about 30% of the time, while larger numbers occur in that position less frequently: 9 as the first digit less than 5% of the time – Benford's Law also concerns the expected distribution for digits beyond the first, which approach a uniform distribution

17. Benford Distribution Between Compressed and Uncompressed Data

18. Benford Distribution Between Control and Event

19. Big Data from Microgrids Complexities introduced by the large amount of multivariate and heterogeneous data streaming from complex sensor networks Extremely large, complex sensor networks, enabling a novel feature reduction method that scales well

20. 20

21. 21