1. Machine SCADA Data Analysis Using MATLAB
ASPEC Team Meeting
"Design Tools and Templates"
Friday, 3 Aug 2018, Sydney
Tatsuya Uno

2. Project Examples Reclaimer EOL fatigue study (W020)
SCADA data  Actual loads and cycles
Strain gauge data  FE model verification/calibration
Reclaimer slew drive alarms investigation (W043k)
SCADA data  Identify pattern of alarms  Establish cause
Engineering Inspections and Plant Integrity Reviews (WR___)
SCADA data  Check operating within design assumptions
 Report on utilisation and productivity
Reclaimer boom dynamics fatigue assessment (W301)
10 Hz SCADA data  Identify patterns
 Quantify loads and cycles  Compare against design

3. MATLAB Introduction and Demonstration
What is MATLAB?
Numerical computing software (compare symbolic)
Programming language
Strengths
Large amounts of data
Numerous mathematical functions
Visualisation tools
Repetitive tasks
Support
Updated, advancements made
Well documented (official and community)
Weaknesses
Learning curve
Cost

4. Demonstration Layout Show example Windows Import data - drag and drop
Directory, command, history, workspace
Show example
Import data - drag and drop
Find example with both numerical and categorical data
Table, timetable - structure, metadata
summary(), head(), tail()
Tab completion
F1 Help
Open
Plot
plot(), histogram()
zoom, pan, data label
Plot controls – title, labels, legend
Generate script
Publish to *.doc

5. SCADA Data RFI

6. SCADA Data RFI (contd.)

7. SCADA Data RFI (contd.)

8. Importing Data into MATLAB - Datastore
Repository for a collection of data files (w/ same structure and formatting)
Designed for data too large to fit in memory (see tall arrays)
How to use
Specify which files form the datastore (raw data)
Specify how a single file from the datastore is to be read
Process one, multiple, or all files at a time
How we use (data not too large to fit in memory)
Load all files from the datastore
Clean up data
Save as a single file (*.mat)

9. Importing Data into MATLAB - Datastore (contd.)
ReadFcn example (fileDatastore 'ReadFcn')

10. Importing Data into MATLAB - Datastore (contd.)
Problem: Datastore file order based on filename
Need to sort data chronologically
Method 1
Read datastore (raw data) once and sort files
Update datastore with chronological order of files
Read from datastore (raw data) again in correct order
Method 2
Import the data in any order then sort all the rows in memory
¿Porque no los dos?

11. Importing Data into MATLAB - Import Functions
CSV format reads much faster than Excel format
Recommend first converting *.xls? files to *.csv outside of MATLAB
Import CSV with
readtable(), with detectImportOptions()
Not recommended
readtable() without detectImportOptions()
Too slow unless 'TreatAsEmpty' known
csvread()
Numeric data only
xlsread()
Too slow with non-numeric data
textscan()
Too low level
Cell arrays use lots of memory

12. Importing Data into MATLAB (contd.)
Achtung!
Excel formatting of CSV files
Saving CSV
Numbers and dates saved with current display format
Lost decimal places, seconds, minutes, hours, etc
Opening CSV
Date formatting applied (may differ from original)
Recommendations
Format dates with required precision or as "General" (serial date)
Use CSVed (freeware), alternatively use
Notepad

13. Summarise Data - Example

14. Importing Data into MATLAB (contd.) - Issues with Data
Summarise data, look for common issues
Inconsistent file structure, format
Large gaps between files, files overlapping in time
Low sampling frequency
Batch plot - More issues
Wrong data
Too much missing data
Erroneous data
Interpolated data (e.g. BW current, position)
Linear
Zero-order hold
Resolution of position data
Zero error (weightometer, cylinder pressures, BW torque arm)
Sign convention
Drive power/torque
Speed %

15. Issues with Data - Examples

16. Issues with Data - Examples

17. Issues with Data - Examples

18. Issues with Data - Examples

19. Issues with Data - Examples

20. Processing Data with MATLAB - Scripts
Run in order.
Results to publish.
Run in any order.
Add new ones.
Read raw data *.csv
Read single *.mat
Read single *.mat

21. MATLAB Scripts and Visualisation Examples
Conveyor strength (F), and fatigue (CF.F)
Utilisation
Boom tip position
Luffing cylinder force
Digging load strength (U), and fatigue (CU.U)
Performance
Points
2D vs 3D plots
Linear vs log scales

22. MATLAB Scripts and Visualisation Examples (contd.)FS FT

23. MATLAB - Recommendations
General
F1 contextual help
Tab completion
Publish scripts to *.doc without code
Use and write custom functions for repetitive tasks - neater scripts
Consider result of NaNs in data for all operations performed
SCADA analysis
Use datastore for reading raw data
Use readtable() with detectImportOptions() for reading *.CSV files
Use timetable (preferred) or table data types
retime()
synchronize()
isregular()
Store metadata via TT.Properties
Standardise naming of timetable variable names
Reference timetable variables by name (not number)

24. Bonus slides from Feb 2016

25. Visualisation
Linkaxes

26. Licencing and Resources
ASPEC MATLAB licences
1×Standalone (Perth)
1×Network license
Signal Processing Toolbox
Optimization Toolbox
MATLAB Documentation
MATLAB Courses
Fundamentals online course notes (screenshots) are available
MATLAB Central (forum)
File Exchange (user written functions and apps/tools)