數值方法 2008, Applied Mathematics NDHU 1  Nonlinear Recursive relations  Nonlinear recursive relations  Time series prediction  Hill-Valley classification  Lorenz series Neural Recursions

數值方法 2008, Applied Mathematics NDHU 2 Nonlinear recursion Post-nonlinear combination of predecessors z[t]=tanh(a 1 z[t-1]+ a 2 z[t-2]+…+ a  z[t-  ])+ e[t], t= ,…,N

數值方法 2008, Applied Mathematics NDHU 3 Post-nonlinear Projection tanh y

數值方法 2008, Applied Mathematics NDHU 4 Data generation z[t]=tanh(a 1 z[t-1]+ a 2 z[t-2]+…+ a  z[t-  ])+ e[t], t= ,…,N tanh

數值方法 2008, Applied Mathematics NDHU 5 Nonlinear Recursion LM learning

數值方法 2008, Applied Mathematics NDHU 6 Prediction Use initial  -1 instances to generate the full time series (red) based on estimated post-nonlinear filter Post-nonlinear filter

數值方法 2008, Applied Mathematics NDHU 7 UCI Machine Learning Repository Nonlinear recursions: hill-valley Classify hill and valley

數值方法 2008, Applied Mathematics NDHU 8 Noise-free Hill Valley

數值方法 2008, Applied Mathematics NDHU 9 Noise-free Hill Valley

數值方法 2008, Applied Mathematics NDHU 10 Hill-Valley with noise

數值方法 2008, Applied Mathematics NDHU 11 Hill-Valley with noise

數值方法 2008, Applied Mathematics NDHU 12 Hill-Valley with noise

數值方法 2008, Applied Mathematics NDHU 13 Nonlinear recursion LM_RBF: z[t]=F(z[t-1], z[t-2], …,z[t-  ])+ e[t], t= ,…,N LM_RBF 1

數值方法 2008, Applied Mathematics NDHU 14 LM_RBF 1 LM_RBF 2

數值方法 2008, Applied Mathematics NDHU 15 Nonlinear Recursion LM learning MLPotts 1 (H)

數值方法 2008, Applied Mathematics NDHU 16 Nonlinear Recursion LM learning MLPotts 2 (V)

數值方法 2008, Applied Mathematics NDHU 17 Approximating error LM_RBF 1 (H) Green: Original Hill Blue: Approximated Hill Red: Approximating error

數值方法 2008, Applied Mathematics NDHU 18 Approximating error LM_RBF 2 (V) Green: Original Hill Blue: Approximated Hill Red: Approximating error

數值方法 2008, Applied Mathematics NDHU 19 Separable 2D diagram of Hill-Valley Noise-free data set 1 LM_RBF 2 (V) LM_RBF 1 (H) Mean Approximating Errors of Two Models

數值方法 2008, Applied Mathematics NDHU 20 Separable 2D diagram of Hill-Valley Noise-free data set 2 Mean Approximating Errors of Two Models LM_RBF 2 (V) LM_RBF 1 (H)

數值方法 2008, Applied Mathematics NDHU 21 Separable 2D diagram of Hill-Valley Dataset 1 (Noise) Mean Approximating Errors of Two Models LM_RBF 2 (V) LM_RBF 1 (H)

數值方法 2008, Applied Mathematics NDHU 22 Separable 2D diagram of Hill-Valley Dataset 2 (Noise) Mean Approximating Errors of Two Models LM_RBF 2 (V) LM_RBF 1 (H)

數值方法 2008, Applied Mathematics NDHU 23 Chaos Time Series Eric's Home Page

數值方法 2008, Applied Mathematics NDHU 24 Lorenz Attractor

數值方法 2008, Applied Mathematics NDHU 25 Lorenz >> NN=20000; >> [x,y,z] = lorenz(NN); >> plot3(x,y,z,'.');

數值方法 2008, Applied Mathematics NDHU 26

數值方法 2008, Applied Mathematics NDHU 27 Nonlinear Recursion Neural nonlinear recursive relations

數值方法 2008, Applied Mathematics NDHU 28 Prediction Use initial  -1 instances to generate the full time series (red) based on derived neural recursions

數值方法 2008, Applied Mathematics NDHU 29 ρ=14, σ=10, β=8/3 ρ=13, σ=10, β=8/3 ρ=15, σ=10, β=8/3ρ=28, σ=10, β=8/3 Rayleigh number

數值方法 2008, Applied Mathematics NDHU 30 Sunspot time series sunpot data load sunspot_train.mat; n=length(Y); plot(1712:1712+n-1,Y);

數值方法 2008, Applied Mathematics NDHU 31 Sunspot time series sunpot data (testing) load sunspot_test_1.mat; n=length(Y); plot(1952:1952+n-1,Y);

數值方法 2008, Applied Mathematics NDHU 32 S[1] S[2] S[3]. S[  -1] S[  ]. S[t-  ] S[t-  +1]. S[t-1] S[t] S[t+1]. x[t] y[t] x[  -1] y[  -1] Time-series 2 paired data

數值方法 2008, Applied Mathematics NDHU 33 aMaM a2a2 a1a1 x Network tanh ……. 1 b2b2 b1b1 bMbM 1 r1r1 r2r2 rMrM r M+1 y

數值方法 2008, Applied Mathematics NDHU 34 Paired data : tau=2 x=[]; len=3; for i=1:n-len p=Y(i:i+len-1)'; x=[x p]; y(i)=Y(i+len); end figure; plot3(x(2,:),x(1,:),y,'.');hold on; y x

數值方法 2008, Applied Mathematics NDHU 35 Learning and prediction M=input('keyin the number of hidden units:'); [a,b,r]=learn_MLP(x',y',M); y=eval_MLP2(x,r,a,b,M); hold on; plot(1712+2: length(y)-1,y,'r') MSE for training data ME for training data K

數值方法 2008, Applied Mathematics NDHU 36 Prediction

數值方法 2008, Applied Mathematics NDHU 37 Sunspot Source codes

數值方法 2008, Applied Mathematics NDHU 38 Paired data: tau=5 x=[]; len=5; for i=1:n-len p=Y(i:i+len-1)'; x=[x p]; y(i)=Y(i+len); end

數值方法 2008, Applied Mathematics NDHU 39 Learning and prediction M=input('keyin the number of hidden units:'); [a,b,r]=learn_MLP(x',y',M); y=eval_MLP2(x,r,a,b,M); hold on; plot(1712+2: length(y)-1,y,'r') ???

數值方法 2008, Applied Mathematics NDHU 40 Example Repeat numerical experiments in slide #34 with Summarize your numerical results in a table

數值方法 2008, Applied Mathematics NDHU 41 RBF y exp x..

數值方法 2008, Applied Mathematics NDHU 42 addpath('..\..\..\Methods\FunAppr\LM_RBF'); % x : Nxd % y : 1xN N=200;d=2; x=rand(N,d)*2-1; y=tanh(-x(:,1).^2-x(:,2).^2); eval(['load data\sunspot_train.mat']); %([ ],:) tx = X'; ty = Y; Net=LM_RBF(tx,ty); [yhat,D]=evaRBF(tx,Net); mean((ty-yhat).^2)/2

數值方法 2008, Applied Mathematics NDHU 43 eval(['load data\sunspot_test_1.mat']); tex1 = X'; tey1 = Y; [yhat,D]=evaRBF(tex1,Net); mean((tey1-yhat).^2)/2 One_step_look_ahead prediction