Download presentation

Presentation is loading. Please wait.

Published byShea Rathmell Modified over 3 years ago

1
Materials Process Design and Control Laboratory Veera Sundararaghavan and Prof. Nicholas Zabaras Materials Process Design and Control Laboratory Sibley School of Mechanical and Aerospace Engineering 188 Frank H. T. Rhodes Hall Cornell University Ithaca, NY 14853-3801 Email: vs85@cornell.edu, zabaras@cornell.eduvs85@cornell.eduzabaras@cornell.edu URL: http://mpdc.mae.cornell.edu/ STATISTICAL LEARNING METHODS FOR MICROSTRUCTURES

2
Materials Process Design and Control Laboratory WHAT IS STATISTICAL LEARNING Statistical learning is all about automating the process of searching for patterns from large scale statistics. Which patterns are interesting? Mathematical techniques for associating input data with desired attributes and identifying correlations A powerful tool for designing materials

3
Materials Process Design and Control Laboratory FOR MICROSTRUCTURES? Properties of a material are affected by the underlying microstructure Microstructural attributes related to specific properties Examples: Correlation functions -> Elastic moduli Orientation distribution ->Yield stress in polycrystals Attributes evolve during processing (thermo mechanical, chemical, solidification etc.) Can we identify specific patterns in these relationships? Is it possible to probabilistically predict the best microstructure and the best processing paths for optimizing properties based on available structural attributes?

4
Materials Process Design and Control Laboratory TERMINOLOGY Microstructure can be represented in terms of typical attributes Examples are volume fractions, probability functions, shape/size attributes, orientation of grains, cluster functions, lineal measures and so on All these attributes affect physical properties Attributes evolve during processing of a microstructure Attributes are represented in a discrete (vector) form as ‘features’ ‘features’ are represented as a vector x k, k = 1,…,n where n is the dimensionality of the feature Every different feature is represented as x k (i) where superscript denotes the i th feature that we are interested in

5
Materials Process Design and Control Laboratory TERMINOLOGY Given a data set of computational or experimental microstructures, can we learn the functional differences between them based on features? We denote microstructures that are similar in attributes in terms of a class representation ‘y’, y = 1..k where k is number of classes. Classes are formed into hierarchies: Each level represented by feature x (i). Structure based classes are affiliated with process and properties: powerful tool for exploring complex microstructure design space

6
Materials Process Design and Control Laboratory APPLICATIONS

7
quantification and mining associations Input microstructure Classifier Feature Detection MICROSTRUCTURE LIBRARIES FOR REPRESENTATION Identify and add new classes Employ lower- order features Pre-processing Sundararaghavan & Zabaras, Acta Materialia, 2004

8
Materials Process Design and Control Laboratory MICROSTRUCTURE RECONSTRUCTION vision Database 2D Imaging techniques Microstructure Analysis (FEM/Bounding theory) Feature extraction Pattern recognition Microstructure evolution models Process Reverse engineer process parameters 3D realizations Sundararaghavan and Zabaras, Computational Materials Sci, 2005

9
Materials Process Design and Control Laboratory Training samples ODF Image Pole figures STATISTICAL LEARNING TOOLBOX Functions: 1.Classification methods 2.Identify new classes NUMERICAL SIMULATION OF MATERIAL RESPONSE 1.Multi-length scale analysis 2.Polycrystalline plasticity PROCESS DESIGN ALGORITHMS 1. Exact methods (Sensitvities) 2.Heuristic methods Update data In the library Associate data with a class; update classes Process controller STATISTICAL LEARNING TOOLBOX

10
Materials Process Design and Control Laboratory DESIGNING PROCESSES FOR MICROSTRUCTURES Process sequence-1 Process parameters ODF history Reduced basis Process sequence-2 New process parameters ODF history Reduced basis Classifier Adaptive basis selection Optimization Reduced basis Process Probable Process sequences & Initial parameters Desired texture/propert y Stage - 1Stage - 2 New dataset added DATABASE Optimum parameters Sundararaghavan and Zabaras, Acta Materialia, 2005

11
Materials Process Design and Control Laboratory THIS LECTURE WILL COVER…. This lecture we will try to go into the math behind statistical learning and learn two really useful techniques – Support Vector Machines and Bayesian Clustering. Applications to microstructure representation, reconstruction and process design will be shown We will skim over the physics and some important computational tools behind these problems

12
Materials Process Design and Control Laboratory STATISTICAL LEARNING TECHNIQUES Regressor Prediction of real-valued output Input Attributes Density Estimator Probability Input Attributes Classifier Prediction of categorical output Input Attributes

13
Materials Process Design and Control Laboratory Regressor Prediction of real-valued output Input Attributes Density Estimator Probability Input Attributes Classifier Prediction of categorical output Input Attributes This lecture STATISTICAL LEARNING TECHNIQUES

14
Materials Process Design and Control Laboratory Regressor Prediction of real-valued output Input Attributes Density Estimator Probability Input Attributes Classifier Prediction of categorical output Input Attributes This lecture Function approximation: Useful for prediction in regions that are computationally unreachable (not covered in this lecture) STATISTICAL LEARNING TECHNIQUES

15
Materials Process Design and Control Laboratory PRELIMINARIES OF SUPERVISED CLASSIFIERS Classifier y x Decision Function y = w.f(x)+b Microstructure classes eg. based on a property Microstructure features denotes +1 denotes -1 Two class problem: The classes for the test specimens are known apriori Aim: To predict the strength of a new microstructure Volume fraction Pore density High strength Low strength

16
Materials Process Design and Control Laboratory SUPPORT VECTOR MACHINES denotes +1 denotes -1 f(x,w,b) = sign(w. x - b) How would you classify this data?

17
Materials Process Design and Control Laboratory OCCAM’S RAZOR plurality should not be assumed without necessity William of Ockham, Surrey (England) 1285-1347 AD, theologian Simpler models are more likely to be correct than complex ones Nature prefers simplicity. principle of uncertainty maximization

18
Materials Process Design and Control Laboratory SUPPORT VECTOR MACHINES denotes +1 denotes -1 f(x,w,b) = sign(w. x - b) How would you classify this data?

19
Materials Process Design and Control Laboratory SUPPORT VECTOR MACHINES denotes +1 denotes -1 f(x,w,b) = sign(w. x - b) Any of these would be fine....but which is best?

20
Materials Process Design and Control Laboratory SUPPORT VECTOR MACHINES denotes +1 denotes -1 f(x,w,b) = sign(w. x - b) Define the margin of a linear classifier as the width that the boundary could be increased by before hitting a datapoint.

21
Materials Process Design and Control Laboratory SUPPORT VECTOR MACHINES denotes +1 denotes -1 f(x,w,b) = sign(w. x - b) The maximum margin linear classifier is the linear classifier with the, um, maximum margin. This is the simplest kind of SVM (Called an LSVM) Linear SVM Support Vectors are those datapoints that the margin pushes up against

22
Materials Process Design and Control Laboratory SUPPORT VECTOR MACHINES Plus-plane = { x : w. x + b = +1 } Plus-plane = { x : w. x + b = +1 } Minus-plane = { x : w. x + b = -1 } Minus-plane = { x : w. x + b = -1 } The vector w is perpendicular to the Plus Plane. Why? The vector w is perpendicular to the Plus Plane. Why? “Predict Class = +1” zone “Predict Class = -1” zone wx+b=1 wx+b=0 wx+b=-1 M = Margin Width How do we compute M in terms of w and b? Let u and v be two vectors on the Plus Plane. What is w. ( u – v ) ? And so of course the vector w is also perpendicular to the Minus Plane Claim: x + = x - + w for some value of. Why? Claim: x + = x - + w for some value of. Why? x+ x-

23
Materials Process Design and Control Laboratory SUPPORT VECTOR MACHINES What we know: What we know: w. x+ + b = +1 w. x+ + b = +1 w. x- + b = -1 w. x- + b = -1 x+ = x- + w x+ = x- + w |x+ - x- | = M |x+ - x- | = M It’s now easy to get M in terms of w and b It’s now easy to get M in terms of w and b “Predict Class = +1” zone “Predict Class = -1” zone wx+b=1 wx+b=0 wx+b=-1 M = Margin Width w. (x - + w) + b = 1 => w. x - + b + w.w = 1 => -1 + w.w = 1 => x-x- x+x+ Computing the margin width

24
Materials Process Design and Control Laboratory SUPPORT VECTOR MACHINES Learning the Maximum Margin Classifier “Predict Class = +1” zone “Predict Class = -1” zone wx+b=1 wx+b=0 wx+b=-1 M = Minimize w.w What are the constraints? w. x k + b >= 1 if y k = 1 w. x k + b <= -1 if y k = -1

25
Materials Process Design and Control Laboratory SUPPORT VECTOR MACHINES denotes +1 denotes -1 This is going to be a problem! What should we do? Minimize w.w + C (distance of error points to their correct place)

26
Materials Process Design and Control Laboratory SUPPORT VECTOR MACHINES wx+b=1 wx+b=0 wx+b=-1 M = 77 11 22 Constraints? w. x k + b >= 1- k if y k = 1 w. x k + b <= -1+ k if y k = -1 k >= 0 for all k Minimize

27
Materials Process Design and Control Laboratory SUPPORT VECTOR MACHINES Harder 1-dimensional dataset What can be done about this? x=0

28
Materials Process Design and Control Laboratory SUPPORT VECTOR MACHINES x=0 Quadratic Basis Functions

29
Materials Process Design and Control Laboratory SUPPORT VECTOR MACHINES WITH KERNELS Constraints? w. (x k )+ b >= 1- k if y k = 1 w. (x k )+ b <= -1+ k if y k = -1 k >= 0 for all k Minimize Φ: x → φ(x)

30
Materials Process Design and Control Laboratory SUPPORT VECTOR MACHINES: QUADRATIC PROGRAMMING Datapoints with k > 0 will be the support vectors Maximize where Subject to these constraints: Then define: Then classify with: f(x,w,b) = sign(w. (x) - b)

31
Materials Process Design and Control Laboratory MULTIPLE CLASSES Class-A Class-B Class-C A C B A B C Given a new microstructure with its ‘s’ features given by find the class of 3D microstructure (y ) to which it is most likely to belong. p = 3 One Against One Method: Step 1: Pair-wise classification, for a p class problem Step 2: Given a data point, select class with maximum votes out of

32
Materials Process Design and Control Laboratory MULTIPLE FEATURES Class - 1 3D Microstructures Class - 2 FEATURE – 1: GRAIN SHAPE FEATURE – 2 GRAIN SIZES Class - 1 Class - 2 Class - 3 Class - 4 Rose of intersections Heyn int. Histogram HIERARCHICAL LIBRARIES – (a.k.a) DIVISIVE CLUSTERING

33
Materials Process Design and Control Laboratory DYNAMIC MICROSTRUCTURE LIBRARY: CONCEPTS Space of all possible microstructures New class New class: partition Expandable class partitions (retraining) Hierarchical sub- classes (eg. medium grains) A class of microstructures (eg. equiaxial grains) Dynamic Representation: Axis for representation New microstructure added Updated representation distance measures

34
Materials Process Design and Control Laboratory QUANTIFICATION OF DIVERSE MICROSTRUCTURE A Common Framework for Quantification of Diverse Microstructure Representation space of all possible polyhedral microstructures Equiaxial grain microstructure space Qualitative representation Lower order descriptor approach Equiax grains Grain size: small Grain size distribution Grain size number No. of grains Quantitative approach1.42.64.00.9….. Microstructure represented by a set of numbers

35
Materials Process Design and Control Laboratory BENEFITS 1.A data-abstraction layer for describing microstructural information. 2.An unbiased representation for comparing simulations and experiments AND for evaluating correlation between microstructure and properties. 3.A self-organizing database of valuable microstructural information which can be associated with processes and properties. Data mining: Process sequence selection for obtaining desired properties Identification of multiple process paths leading to the same microstructure Adaptive selection of basis for reduced order microstructural simulations. Hierarchical libraries for 3D microstructure reconstruction in real-time by matching multiple lower order features. Quality control: Allows machine inspection and unambiguous quantitative specification of microstructures.

36
Materials Process Design and Control Laboratory PRINCIPAL COMPONENT ANALYSIS Let be n images. 1. Vectorize input images 2. Create an average image 3. Generate training images 4. Create correlation matrix (L mn ) 5. Find eigen basis (v i ) of the correlation matrix 6. Eigen microstructures (u i ) are generated from the basis (v i ) as 7. Any new face image ( ) can be transformed to eigen face components through ‘n’ coefficients (w k ) as, Representation coefficients Reduced basis Data Points

37
Materials Process Design and Control Laboratory REQUIREMENTS OF A REPRESENTATION SCHEME REPRESENTATION SPACE OF A PARTICULAR MICROSTRUCTURE Need for a technique that is autonomous, applicable to a variety of microstructures, computationally feasible and provides complete representation A set of numbers which completely represents a microstructure within its class2.73.61.20.1….. 8.42.15.71.9….. Must differentiate other cases: (must be statistically representative)

38
Materials Process Design and Control Laboratory PCA REPRESENTATION OF MICROSTRUCTURE – AN EXAMPLE Eigen-microstructures Input Microstructures Representation coefficients (x 0.001) Image-1 quantified by 5 coefficients over the eigen- microstructures 0.01251.3142-4.234.5429-1.6396 -0.84060.8463-3.02320.34242.6752 3.943-4.2162-0.6817-97181.9268 1.1796-1.3354-2.84016.2064-3.2106 5.82945.2287-3.7972-3.6095-3.6515 Basis 5 Basis 1

39
Materials Process Design and Control Laboratory EIGEN VALUES AND RECONSTRUCTION OVER THE BASIS 1.Reconstruction with 100% basis 2. Reconstruction with 80% basis 3. Reconstruction with 60% basis 4. Reconstruction with 40% basis 4231 Reconstruction of microstructures over fractions of the basis Significant eigen values capture most of the image features

40
Materials Process Design and Control Laboratory INCREMENTAL PCA METHOD For updating the representation basis when new microstructures are added in real-time. Basis update is based on an error measure of the reconstructed microstructure over the existing basis and the original microstructure IPCA : Given the Eigen basis for 9 microstructures, the update in the basis for the 10 th microstructure is based on a PCA of 10 x 1 coefficient vectors instead of a 16384 x 1 size microstructures. Updated Basis Newly added data point

41
Materials Process Design and Control Laboratory ROSE OF INTERSECTIONS FEATURE – ALGORITHM (Saltykov, 1974) Identify intercepts of lines with grain boundaries plotted within a circular domain Count the number of intercepts over several lines placed at various angles. Total number of intercepts of lines at each angle is given as a polar plot called rose of intersections

42
Materials Process Design and Control Laboratory GRAIN SHAPE FEATURE: EXAMPLES

43
Materials Process Design and Control Laboratory GRAIN SIZE PARAMETER Several lines are superimposed on the microstructure and the intercept length of the lines with the grain boundaries are recorded (Vander Voort, 1993) The intercept length (x-axis) versus number of lines (y-axis) histogram is used as the measure of grain size.

44
GRAIN SIZE FEATURE: EXAMPLES Materials Process Design and Control Laboratory

45
SVM TRAINING FORMAT CLASSIFICATION SUCCESS % Total images Number of classes Number of Training images Highest success rate Average success rate 375114095.8292.53 3751110098.5495.80 Class Feature number Feature value Feature number Feature value 1123.32221.52 2124.12231.52 Data point GRAIN FEATURES: GIVEN AS INPUT TO SVM TRAINING ALGORITHM

46
Materials Process Design and Control Laboratory CLASS HIERARCHY Class –2 Class –1 Class 1(a)Class 1(b)Class 1(c)Class 2(a)Class 2(b)Class 2(c) Level 1 : Grain shapes Level 2 : Subclasses based on grain sizes New classes: Distance of image feature from the average feature vector of a class

47
IPCA QUANTIFICATION WITHIN CLASSES Materials Process Design and Control Laboratory Class-j Microstructures (Equiaxial grains, medium grain size) Class-i Microstructures (Elongated 45 degrees, small grain size) Representation Matrix Image -1Image-2Image-3… Component in basis vector 1 1232338 29154-85 3-549012 Average Image 212324 … Eigen Basis 0.90.840.23.. 0.540.210.74.. The Library – Quantification and image representation

48
Materials Process Design and Control Laboratory REPRESENTATION FORMAT FOR MICROSTRUCTURE Improvement of microstructure representation due to classification Date: 1/12 02:23PM, Basis updated Shape Class: 3, (Oriented 40 degrees, elongated) Size Class : 1, (Large grains) Coefficients in the basis:[2.42, 12.35, -4.14, 1.95, 1.96, -1.25] Reconstruction with 6 coefficients (24% basis): A class with 25 images Improvement in reconstruction: 6 coefficients (10 % of basis) Class of 60 images Original image Reconstruction over 15 coefficients

49
Materials Process Design and Control Laboratory MICROSTRUCTURE REPRESENTATION USING SVM & PCA COMMON-BASIS FOR MICROSTRUCTURE REPRESENTATION Does not decay to zero A DYNAMIC LIBRARY APPROACH Classify microstructures based on lower order descriptors. Create a common basis for representing images in each class at the last level in the class hierarchy. Represent 3D microstructures as coefficients over a reduced basis in the base classes. Dynamically update the basis and the representation for new microstructures

50
Materials Process Design and Control Laboratory PCA MICROSTRUCTURE RECONSTRUCTION Pixel value round-off Basis Components X 5.89 X 14.86 + Project onto basis Reconstruct using two basis components Representation using just 2 coefficients (5.89,14.86)

51
Materials Process Design and Control Laboratory MOTIVATION 1.Creation of 3D microstructure models from 2D images 2. 3D imaging requires time and effort. Need to address real–time methodologies for generating 3D realizations. 3. Make intelligent use of available information from computational models and experiments. vision Database Pattern recognition Microstructure Analysis 2D Imaging techniques

52
Materials Process Design and Control Laboratory LITERATURE: STOCHASTIC MICROSTRUCTURE RECONSTRUCTION Methods available are optimization based: Features of 2D image are matched to that of a 3D microstructure by posing an optimization problem. 1) Does not make use of available information (experimental/simulated data) 2) Cannot perform reconstructions in real-time. Need to take into account the processes that create these microstructure (Oren and Bakke, 2003) for correctly modeling the geometric connectivity. Key assumptions employed for 3D image reconstruction from a single 2D image Randomness Assumption (Ohser and Mucklich – 2000). 1. Grains in a polyhedral microstructure are assumed to be of the similar shapes but of different sizes. 2. Two phase microstructures can be characterized using rotationally-invariant probability functions

53
Materials Process Design and Control Laboratory PATTERN RECOGNITION (PR) STEPS DATABASE CREATION FEATURE EXTRACTION TRAINING PREDICTION Datasets: microstructures from experiments or physical models Extraction of statistical features from the database Creation of a microstructure class hierarchy: Classification methods Prediction of 3D reconstruction, process paths, etc, PATTERN RECOGNITION : A DATA-DRIVEN OPTIMIZATION TOOL Feature matching for reconstruction of 3D microstructures Real-time

54
Materials Process Design and Control Laboratory POLYHEDRAL MICROSTRUCTURES: MC MODEL Potts Hamiltonian (H) Algorithm (1 Monte Carlo Step): Calculation of the free energy of a randomly selected node (H i ) Random choice of a new crystallographic orientation for the node New calculation of the free energy of the element (H f ) The orientation that minimizes the energy (min(H f,H i )) is chosen. N s : Total No. of nodes N n (i) : No. of neighbors of node ‘i’ Microstructure Database Classes of microstructures based on grain size feature

55
Materials Process Design and Control Laboratory POLYHEDRAL MICROSTRUCTURES : GRAIN SIZE FEATURE Intercept lengths of parallel network of lines with the grain boundaries are recorded at several angles The intercept length (x-axis) versus number of lines (y-axis) histogram is the measure of grain size (Heyn intercept histogram). Slice

56
Materials Process Design and Control Laboratory FEATURE BASED CLASSIFICATION Class - 1 3D Microstructures Class - 2 LEVEL - 1 LEVEL - 2 Class - 1 Class - 2 Class - 3 Class - 4 Rose of intersections Heyn int. Histogram

57
Materials Process Design and Control Laboratory RECONSTRUCTION OF POLYHEDRAL MICROSTRUCTURE Polarized light micrographs of Aluminum alloy AA3002 representing the rolling plane (Wittridge & Knutsen 1999) A reconstructed 3D image Comparison of the average feature of 3D class and the 2D image

58
Materials Process Design and Control Laboratory STEREOLOGICAL ESTIMATES OF 3D GRAIN SIZES The stereological integral equation for estimating the 3D grain size distribution from a 2D image for polyhedral microstructures N a,F a (s) : density of grains and grain size distribution in 2D image N v,F v (u) : density of grains and grain size distribution in 3D microstructure : rotation average of the size of a particle with maximum size = 1 G u (s): Size distribution function of the section profiles under the condition that a random size ‘U’ equals the 3D particle mean size (u). Remark: Sizes are defined as the maximum calliper diameter of a grain Numerical Scheme: Let Then : Mean number/volume of grains with size u i Let y k be the mean number per unit area of section profiles with size between [s k-1,s k ] And let u i = a i and s k = a k, then, y = P where P is a matrix formed from a set of coefficients ( based on the shape assumption of grains

59
Materials Process Design and Control Laboratory STEREOLOGICAL DISTRIBUTIONS (GEOMETRICAL) 3D reconstruction 2D grain profile 3D grain 3D grain size distribution based on assumption that particles are randomly oriented cubes ( ) N a,F a (s) : density of grains and grain size distribution in 2D image N v,F v (u) : density of grains and grain size distribution in 3D microstructure

60
Materials Process Design and Control Laboratory STATISTICAL CORRELATION MEASURES MC Sampling: Computing the three point probability function of a 3D microstructure(40x40x40 mic) S 3 (r,s,t), r = s = t = 2, 5000 initial points, 4 samples at each initial point. Rotationally invariant probability functions (S i N ) can be interpreted as the probability of finding the N vertices of a polyhedron separated by relative distances x 1, x 2,..,x N in phase i when tossed, without regard to orientation, in the microstructure.

61
When a voxel solidifies, liquid is expelled to its neighbors, creating solute concentration (c i,j,k ) gradients. Movement of solute to minimize concentration gradients is modeled using fick’s law Materials Process Design and Control Laboratory MC MODEL FOR TWO-PHASE MICROSTRUCTURES Weights (w) of neighbors Face neighbors = 1 Edge neighbors = Solid voxels Microstructure is represented using voxels. Probability of solidification (P) depends on 1) Net weight (w) of the No. of neighbors of a solid voxel: If w >= 8.6568: voxel solidifies (P = 1) If 3.8284 < w < 8.6568, P = 0.1 If weight < 3.8284, the voxel remains liquid (P = 0) 2) The solute concentration: A linear probability distribution with P = 0 at critical concentration and P = 1 when concentration is 0. Final state Where (i,j,k) is a voxel coordinate, n is the time step and D is the diffusion coefficient

62
Materials Process Design and Control Laboratory TWO PHASE MICROSTRUCTURE: CLASS HIERARCHY Class - 1 3D Microstructures Feature vector : Three point probability function 3D Microstructures Class - 2 Feature: Autocorrelation function LEVEL - 1 LEVEL - 2 r m

63
Materials Process Design and Control Laboratory EXAMPLE: 3D RECONSTRUCTION USING SVMS Ag-W composite (Umekawa 1969) A reconstructed 3D microstructure 3 point probability function Autocorrelation function

64
Materials Process Design and Control Laboratory MICROSTRUCTURE ELASTIC PROPERTIES 3D image derived through pattern recognition Experimental image

65
Materials Process Design and Control Laboratory WHAT IS MICROSTRUCTURE DESIGN Initial microstructure processing sequence? Final microstructure/ property Microstructure? Known operating conditions Known property limits Initial Microstructure Known operating conditions Property? Direct problem Design problems Use finite elements, experiments etc. Design for best microstructure Design for best processes

66
Materials Process Design and Control Laboratory SUPERVISED VS UNSUPERVISED LEARNING Supervised classification for design: 1.Classify microstructures based on known process sequence classes 2.Given a desired microstructure, identify the processing stages required through classification 3.Drawback: Identifies a unique process sequence, but we that find many processing paths to lead to similar properties! UNSUPERVISED CLASSIFICATION 1.Identify classes purely based on structural attributes 2.Associate processes and properties through databases 3.Explores the structural attribute space for similarities and unearths non-unique processing paths leading to similar microstructural properties

67
Materials Process Design and Control Laboratory K MEANS Suppose the coordinates of points drawn randomly from this dataset are transmitted. You can install decoding software at the receiver. You’re only allowed to send two bits per point. It’ll have to be a “lossy transmission”. Loss = Sum Squared Error between decoded coords and original coords. What encoder/decoder will lose the least information?

68
Materials Process Design and Control Laboratory K MEANS Idea One 00 1110 01 Break into a grid, decode each bit-pair as the middle of each grid-cell Questions What are we trying to optimize? What are we trying to optimize? Are we sure it will find an optimal clustering? Are we sure it will find an optimal clustering? Break into a grid, decode each bit-pair as the centroid of all data in that grid-cell

69
Materials Process Design and Control Laboratory K MEANS Find the cluster centers {C 1,C 2,…,C k } such that the sum of the 2-norm distance squared between each feature x i, i = 1,..,n and its nearest cluster center c h is minimized. Cost Function = Cost function minimized by transmitting centroids

70
Materials Process Design and Control Laboratory THE EXPECTATION-MAXIMIZATION (EM) ALGORITHM What properties can be changed for centers c 1, c 2, …, c k have when distortion is not minimized? Expectation step: Compute expected centers (1)Change encoding so that x i is encoded by its nearest center Maximization step: Compute maximum likelihood values of centers (2) Set each Center to the centroid of points it owns. There’s no point applying either operation twice in succession. But it can be profitable to alternate. …And that’s K-means! EM algorithm will be dealt with later

71
Materials Process Design and Control Laboratory K-MEANS 1.Ask user how many clusters they’d like. (e.g. k=5)

72
Materials Process Design and Control Laboratory K-MEANS 1.Ask user how many clusters they’d like. (e.g. k=5) 2.Randomly guess k cluster Center locations

73
Materials Process Design and Control Laboratory K-MEANS 1.Ask user how many clusters they’d like. (e.g. k=5) 2.Randomly guess k cluster Center locations 3.Each datapoint finds out which Center it’s closest to. (Thus each Center “owns” a set of datapoints)

74
Materials Process Design and Control Laboratory K-MEANS 1.Ask user how many clusters they’d like. (e.g. k=5) 2.Randomly guess k cluster Center locations 3.Each datapoint finds out which Center it’s closest to. 4.Each Center finds the centroid of the points it owns

75
Materials Process Design and Control Laboratory K-MEANS 1.Ask user how many clusters they’d like. (e.g. k=5) 2.Randomly guess k cluster Center locations 3.Each datapoint finds out which Center it’s closest to. 4.Each Center finds the centroid of the points it owns… 5.…and jumps there 6.…Repeat until terminated! often unknown (is dependent on the features used for microstructure representation)

76
Materials Process Design and Control Laboratory SHORTCOMINGS OF K-MEANS AND REMEDIES 1)K-MEANS gives hyper-spherical clusters: Not always the case with data 2)Number of classes must be known apriori: Beats the reasoning for unsupervised clusters – we do not know anything about the classes in the data 3)May converge to local optima – not so bad We will discuss about new strategies to get improved clusters of microstructural features 1)Gaussian mixture models and Bayesian clustering 2)Later, an improved k-means algorithm called X-means which uses a Bayesian information criterion

77
Materials Process Design and Control Laboratory PROBABILITY PRELIMINARIES A is a Boolean-valued random variable if A denotes an event, and there is some degree of uncertainty as to whether A occurs. Examples A = You win the toss A = Probability of failure of a structure 0 <= P(A) <= 1 P(True) = 1 P(False) = 0 P(A or B) = P(A) + P(B) - P(A and B) P(~A) + P(A) = 1 P(B) = P(B ^ A) + P(B ^ ~A) Discrete Random Variables

78
Materials Process Design and Control Laboratory PROBABILITY PRELIMINARIES P(A ^ B) P(A|B) = ----------- P(B) P(A ^ B) = P(A|B) P(B) P(A ^ B) P(A|B) P(B) P(A ^ B) P(A|B) P(B) P(B|A) = ----------- = --------------- P(A) P(A) P(A) P(A) Corollary: The Chain Rule Definition of Conditional Probability Bayes Rule

79
Materials Process Design and Control Laboratory PROBABILITY PRELIMINARIES MAP (Maximum A-Posteriori Estimator): What if Y = v itself is very unlikely? MLE (Maximum Likelihood Estimator): MLE (Maximum Likelihood Estimator): Includes P(Y = v) information through Bayes rule (P(Y = v) is called as ‘prior’) Class of data = argmax i P(data | class = i) Class of data = argmax i P(class = i | data)

80
Materials Process Design and Control Laboratory PROBABILITY PRELIMINARIES MAP (Maximum A-Posteriori Estimator):

81
Materials Process Design and Control Laboratory PROBABILITY PRELIMINARIES Bayes Classifiers in a nutshell 1. Learn the distribution over inputs for each value Y. 2. This gives P(X 1, X 2, … X m | Y=v i ). 3. Estimate P(Y=v i ). as fraction of records with Y=v i. 4. For a new prediction:

82
Materials Process Design and Control Laboratory NAÏVE BAYES CLASSIFIER In the case of the naive Bayes Classifier this can be simplified: The independent features assumption

83
Materials Process Design and Control Laboratory Notation change: The naïve Bayes classifier New Bayes classifier NAÏVE BAYES CLASSIFIER IS AN SVM?

84
Materials Process Design and Control Laboratory Bayes classifier with feature weighting NAÏVE BAYES CLASSIFIER IS AN SVM? w j = 1 (for naïve Bayes) But, features may be correlated! A two class classifier Decision function given by the sign of f WBC given by Class t Class f

85
Materials Process Design and Control Laboratory NAÏVE BAYES CLASSIFIER IS AN SVM? Class t Class f SVM classifier! Feature space of a naïve Bayes classifier

86
Materials Process Design and Control Laboratory INTRO TO BAYESIAN UNSUPERVISED CLASSIFICATION Gaussian Mixture Models Assume that each feature is generated as: Pick a class at random. Choose class i with probability P(w i ). The feature is sampled from a Gaussian distribution : N( i, i )

87
Materials Process Design and Control Laboratory GAUSSIAN MIXTURE MODEL 11 33 There are k components. The i’th component is called y i Component y i has an associated mean vector i Each component generates data from a Gaussian with mean i and covariance matrix i 22 11 Assuming features in each class can be modeled by a Gaussian distribution, identify the parameters (means,variances etc.) of the distributions Probabilistic extension of K-MEANS

88
Materials Process Design and Control Laboratory GAUSSIAN MIXTURE MODEL We have x 1 x 2 … x n features of a microstructure We have P(y 1 ).. P(y k ). We have σ. We can define, for any x, P(x|y i, μ 1, μ 2.. μ k ) Can we define P(x | μ 1, μ 2.. μ k ) ? Can we define P(x 1, x 2,.. x n | μ 1, μ 2.. μ k ) ?

89
Materials Process Design and Control Laboratory GAUSSIAN MIXTURE MODEL Given a guess at μ 1, μ 2.. μ k, We can obtain the probability of the unlabeled data given those μ‘s. Inverse Problem: Find ’s given the points x 1, x 2,…x k The normal max likelihood trick: Set d log Prob (….) = 0 d μ i and solve for μ i ‘s. Using gradient descent, Slow but doable Use a much faster and recently very popular method…

90
Materials Process Design and Control Laboratory EM ALGORITHM REVISITED We have unlabeled microstructural features x 1 x 2 … x R We know there are k classes We know there are k classes We know P(y 1 ), P(y 2 ), P(y 3 ), …, P(y k ) We know P(y 1 ), P(y 2 ), P(y 3 ), …, P(y k ) We don’t know μ 1 μ 2.. μ k We don’t know μ 1 μ 2.. μ k We can write P( data | μ 1 …. μ k ) We can write P( data | μ 1 …. μ k ) Maximize this likelihood

91
Materials Process Design and Control Laboratory GAUSSIAN MIXTURE MODEL This is n nonlinear equations in μ j ’s.” If, for each x i we knew that for each y j the prob that μ j was in class y j is P(y j |x i,μ 1 …μ k ) Then… we would easily compute μ j. If we knew each μ j then we could easily compute P(y j |x i,μ 1 …μ j ) for each y j and x i.

92
Materials Process Design and Control Laboratory GAUSSIAN MIXTURE MODEL Iterate. On the t’th iteration let our estimates be { μ 1 (t), μ 2 (t) … μ c (t) } E-step E-step Compute “expected” classes of all datapoints for each class M-step. Compute Max. like μ given our data’s class membership distributions Just evaluate a Gaussian at x k

93
Materials Process Design and Control Laboratory GAUSSIAN MIXTURE MODEL: DENSITY ESTIMATION Features in 2D Complex PDF of the feature space Classification + Probabilistic quantification of results Ambiguity + Anomaly detection – Very popular in Genome mapping

94
Materials Process Design and Control Laboratory DATABASE FOR POLYCRYSTAL MICROSTRUCTURES Statistical Learning Feature Extraction Multi-scale microstructure evolution models Process design for desired properties RD R-value TD Meso-scale database COMPONENTS TD Youngs Modulus RD Database Divisive Clustering Class hierarchies Class Prediction Driven by distance based (or) Probabilistic clustering

95
Materials Process Design and Control Laboratory DATABASE FOR POLYCRYSTAL MICROSTRUCTURES Statistical Learning Feature Extraction Multi-scale microstructure evolution models Process design for desired properties RD R-value TD Meso-scale database COMPONENTS TD Youngs Modulus RD Database Divisive Clustering Class hierarchies Class Prediction Cluster based on similar microstructural features

96
Materials Process Design and Control Laboratory DATABASE FOR POLYCRYSTAL MICROSTRUCTURES Statistical Learning Feature Extraction Multi-scale microstructure evolution models Process design for desired properties RD R-value TD Meso-scale database COMPONENTS TD Youngs Modulus RD Database Divisive Clustering Class hierarchies Class Prediction Associate process/ property info from database Cluster based on similar microstructural features

97
Materials Process Design and Control Laboratory ORIENTATION DISTRIBUTION FUNCTION Any macroscale property can be expressed as an expectation value if the corresponding single crystal property χ (,t) is known. Determines the volume fraction of crystals within a region R' of the fundamental region R Probability of finding a crystal orientation within a region R' of the fundamental region Characterizes texture evolution ORIENTATION DISTRIBUTION FUNCTION – A(r,t) – reorientation velocity ODF EVOLUTION EQUATION – EULERIAN DESCRIPTION

98
Materials Process Design and Control Laboratory FEATURES OF AN ODF: ORIENTATION FIBERS Points (r) of a (h,y) fiber in the fundamental region angle Crystal Axis = h Sample Axis = y Rotation (R) required to align h with y (invariant to, ) Fibers: h{1,2,3}, y || [1,0,1] {1,2,3} Pole Figure Point y (1,0,1) Integrated over all fibers corresponding to crystal direction h and sample direction y For a particular (h), the pole figure takes values P(h,y) at locations y on a unit sphere.

99
Materials Process Design and Control Laboratory SIGNIFICANCE OF ORIENTATION FIBERS Uniaxial (z-axis) Compression Texture z-axis fiber BB’ z-axis fiber AA’ z-axis fiber CC’ Predictable fiber development Important fiber families: : uniaxial compression, plane strain compression and simple shear. : Torsion,, fibers: Tension fiber (ND ) & fiber: FCC metals under plane strain compression close affiliation with processes

100
Materials Process Design and Control Laboratory LIBRARY FOR TEXTURES [110] fiber family DATABASE OF ODFs Uni-axial (z-axis) Compression Texture z-axis fiber (BB’) Feature: fiber path corresponding to crystal direction h and sample direction y

101
Materials Process Design and Control Laboratory SUPERVISED CLASSIFICATION USING SUPPORT VECTOR MACHINES Given ODF/texture Tension (T) Stage 1 LEVEL – 2 CLASSIFICATION Plane strain compression T+P LEVEL – I CLASSIFICATION Tension identified Stage 2 Stage 3 Multi-stage classification with each class affiliated with a unique process Identifies a unique processing sequence: Fails to capture the non-uniqueness in the solution

102
Materials Process Design and Control Laboratory UNSUPERVISED CLASSIFICATION Find the cluster centers {C 1,C 2,…,C k } such that the sum of the 2-norm distance squared between each feature x i, i = 1,..,n and its nearest cluster center C h is minimized. Identify clusters Clusters DATABASE OF ODFs Feature Space Cost function Each class is affiliated with multiple processes

103
Materials Process Design and Control Laboratory ODF CLASSIFICATION Desired ODF Search path Automatic class-discovery without class labels. Hierarchical Classification model Association of classes with processes, to facilitate data-mining Can be used to identify multiple process routes for obtaining a desired ODF Data-mining for Process information with ODF Classification ODF 2,12,32,97 One ODF, several process paths

104
Materials Process Design and Control Laboratory PROCESS PARAMETERS LEADING TO DESIRED PROPERTIES Young’s Modulus (GPa) Angle from rolling direction CLASSIFICATION BASED ON PROPERTIES Class - 1 Class - 2 Class - 3 Class - 4 Velocity Gradient Different processes, Similar properties Database for ODFs Property Extraction ODF Classification Identify multiple solutions

105
Materials Process Design and Control Laboratory K-MEANS ALGORITHM FOR UNSUPERVISED CLASSIFICATION User needs to provide ‘k’, the number of clusters. Lloyds Algorithm: 1.Start with ‘k’ randomly initialized centers 2.Change encoding so that x i is owned by its nearest center. 3.Reset each center to the centroid of the points it owns. Alternate steps 1 and 2 until converged. But, No. of clusters is unknown for the texture classification problem

106
Materials Process Design and Control Laboratory SCHWARZ CRITERION FOR IDENTIFYING NUMBER OF CLUSTERS The number of clusters chosen maximizes the Bayesian information criterion given by: Where is the is the log-likelihood of the data taken at the maximum likelihood point, p is the number of free parameters in the model Maximum likelihood of the variance assuming Gaussian data distribution Probability of a point in cluster i Log-likelihood of the data in a cluster

107
Materials Process Design and Control Laboratory CENTROID SPLIT TESTS X-MEANS algorithm: Start with k clusters found through k-means algorithm Split each centroid into two centroids, and move the new centroids along a distance proportional to the cluster size in an arbitrarily chosen direction Run local k-means (k = 2) in each cluster Accept split cluster in each region if BIC(k = 1) < BIC(k = 2) Test for various initial values of ‘k’ and select the ‘k’ with maximum overall BIC Split centersRun local k-means (k = 2) in each cluster New clusters based on BIC

108
Materials Process Design and Control Laboratory COMPARISON OF K-MEANS AND X-MEANS Local Optimum produced by the kmeans algorithm with k = 4 Cluster configuration produced by k-means with k = 6: Over- estimates the natural number of clusters Configuration produced by the x-means algorithm: Input range of k = 2 to 15. x- means found 4 clusters from the data-set based on the Bayesian Information Criterion

109
Materials Process Design and Control Laboratory MULTIPLE PROCESS ROUTES Desired Young’s Modulus distribution Magnetic hysteresis loss distribution Stage: 1 Shear-1 = 0.9580 Stage: 2 Plane strain compression ( = -0.1597 ) Stage: 1 Shear -1 = 0.9454 Stage: 2 Rotation-1 ( = -0.2748) Stage 1: Tension = 0.9495 Stage 2: Shear-1 = 0.3384 Stage 1: Tension = 0.9699 Stage 2: Rotation-1 = -0.2408 Classification

110
Materials Process Design and Control Laboratory LIMITATIONS OF STATISTICAL LEARNING BASED DESIGN SOLUTIONS Classification alone does not yield the final design solution Why? Since it is impossible to explore the infinite design space within a database of reasonable size. Use statistical learning for providing initial class of solutions Use local optimization schemes (details not given in this presentation) to identify the exact solutions Response surface Objective to be minimized Microstructure attributes Stat Learning Design solutions

111
Materials Process Design and Control Laboratory DESIGN FOR DESIRED ODF: A MULTI STAGE PROBLEM Initial guess, = 0.65, = -0.1 Desired ODFOptimal- Reduced order control Full order ODF based on reduced order control parameters Stage: 1 Plane strain compression ( = 0.9472) Stage: 2 Compression ( = -0.2847)

112
Materials Process Design and Control Laboratory DESIGN FOR DESIRED MAGNETIC PROPERTY h Crystal direction. Easy direction of magnetization – zero power loss External magnetization direction Stage: 1 Shear – 1 ( = 0.9745) Stage: 2 Tension ( = 0.4821)

113
Materials Process Design and Control Laboratory DESIGN FOR DESIRED YOUNGS MODULUS Stage: 1 Shear ( = - 0.03579) Stage: 2 Tension ( = 0.17339) Stiffness of F.C.C Cu in crystal frame Elastic modulus is found using the polycrystal average over the ODF as,

114
Materials Process Design and Control Laboratory WHAT WE SHOULD KNOW How to “learn” microstructure/process/property relationships given computational and experimental data Be happy with probabilistic tools: Bayesian analytics and Gaussian mixture models Understand simple tools like K-MEANS that can be readily used. Understand SVMs as a versatile statistical learning tool: For both feature selection and classification Apply statistical learning to perform real-time decisions under high degrees of uncertainty Appreciate the uses and understand the limitations of statistical learning applied to materials

115
Materials Process Design and Control Laboratory USEFUL REFERENCES Andrew Moore’s Statistical learning course online: http://www-2.cs.cmu.edu/~awm/tutorials/ Books: R.O. Duda, P.E. Hart and D.G. Stork, Pattern classification (2nd ed), John Wiley and Sons, New York (2001). Example papers on microstructure/materials related applications for the tools presented in this talk: V. Sundararaghavan and N. Zabaras, "A dynamic material library for the representation of single phase polyhedral microstructures", Acta Materialia, Vol. 52/14, pp. 4111-4119, 2004 V. Sundararaghavan and N. Zabaras, "Classification of three-dimensional microstructures using support vector machines", Computational Materials Science, Vol. 32, pp. 223-239, 2005 V. Sundararaghavan and N. Zabaras, "On the synergy between classification of textures and deformation process sequence selection", Acta Materialia, Vol. 53/4, pp. 1015-1027, 2005 T J Sabin, C A L Bailer-Jones and P J Withers, Accelerated learning using Gaussian process models to predict static recrystallization in an Al–Mg alloy, Modelling Simul. Mater. Sci. Eng. 8 (2000) 687–706 C. A. L. Bailer-Jones, H. K. D. H. Bhadeshia and D. J. C. MacKay, Gaussian Process Modelling of Austenite Formation in Steel, Materials Science and Technology, Vol. 15, 1999, 287-294.,

116
Materials Process Design and Control Laboratory THANK YOU

Similar presentations

OK

ECE 8443 – Pattern Recognition ECE 8527 – Introduction to Machine Learning and Pattern Recognition Objectives: Mixture Densities Maximum Likelihood Estimates.

ECE 8443 – Pattern Recognition ECE 8527 – Introduction to Machine Learning and Pattern Recognition Objectives: Mixture Densities Maximum Likelihood Estimates.

© 2018 SlidePlayer.com Inc.

All rights reserved.

Ads by Google

Ppt on power generation using footsteps dance Ppt on regular expression syntax Ppt on service industry in india Ppt on land soil water natural vegetation and wildlife resources Ppt on power generation by coal Ppt on market friendly state political cartoon How to view ppt online Ppt on obesity prevention policy Vlsi physical design ppt on high level synthesis Ppt on operating system architecture