1. Predicting Post-Operative Gait of Cerebral Palsy Patients
Movement Research Lab.
Seoul National University

2. Motivation
We want to predict the gait of post-operative patients to compensate doctors’ experience.
정형외과 수술의 효과가 즉각적으로 나타나지 않기 때문에 수술 후 환자의 보행이 어떠할지 예측하기 어렵다. 따라서 의사의 경험과 직관에 의존하여 수술에 대한 판단을 하게 된다. 우리는 수술 후 환자 동작을 예측하여 이러한 판단에 보완이 되도록 하기 위해 본 연구를 진행하였다.
//우리는 수술 후 환자 보행이 어떻게 바뀔지를 예측하여 수술을 하는 것이 도움이 될지 아니면 고통만 가중되고 힘들지만 할지를 알아보고//자 한다.
//어떤 수술을 할지 근육, 뼈 이런 판단을 예전에는 경험에 의해서만 했지만 수량화된 값으로 제시해주면

3. Our goal Predicting post-operative gait
learning from pre & post operative gait data

4. Motion predictor Learn a motion predictor from training data set .
- : pre-operative gait (input)
- : post-operative gait (output)
Given new input data, we generate a new motion using the learned predictor.
New pre-operative gait, x
새로운 input data, 수술 전 보행 데이터가 시스템에 들어왔을 때 새로운 수술 후 보행 결과는 prediction model을 통해서 만들어진다.
Prediction model은 학습 데이터로부터 regression 과정을 통해서 만들어진다.
New post-
operative
gait
Training
data
Regression
process
Predictor

5. Regression process . . . . . . Pre-operative gait i Regression
Motion to motion i
Regression 과정을 좀 더 살펴보겠습니다. 한 명의 수술 전 환자의 gait사이클은 다음과 같은 자세들로 구성되어 있습니다. 마찬가지로
같은 환자의 수술 후 gait 사이클은 다음과 같은 자세들로 구성되어 있습니다. 수술 전후의 여러 명의 환자의 gait를 이용하여 regression을 수행하는데요 우선, Regression을 적용하기 위해 수술 전 후 환자데이터를 벡터화 해야 하는데요. 수술 전 후 환자 자세로부터 모든 관절의 방향 값을 얻어낼 수 있고 이 값들을 쌓아서 벡터화 합니다. i 번째 수술 전 환자의 전체 보행 모션 Xi 에서 i 번째 수술 후 환자 전체 보행 모션 Yi을 motion-to-motion 변환 방법을 기반으로 regression을 수행을 합니다.
. . .
Post-operative gait

6. Canonical Correlation Analysis (CCA)
Find pairs of basis that maximize the correlation between two variables in the reduced space.
Variable X
Regression
Reduced Y
Projection
Basis X,
CCA
Variable Y
Basis Y
Reduced X
With such a training data in hand, we consider canonical correlation analysis, CCA as a good prediction model/ because CCA well explain data dependency between input and output.
So CCA can minimize the prediction error./ CCA finds pairs of basis that maximize the correlation between two variables x & y in subspace. / When we perform the regression in the reduced space, the fitting errors are minimized because two variables are highly correlated in the reduced space./ X bar and Y bar represent the reduced variable x and y, respectively. / After some substitution, we can define the CCA equation /The pair basis which is solution of CCA are obtained by singular value decomposition.
//Please see our paper for the details of the derivation of this equation.
// We perform the regression in the reduces pace.
Correlation :

7. Sparse CCA
Reformulation

8. CCA-based regression Linear regression between pair of reduced data
Reconstruction from subspace to original space
Concatenating these matrix produces the predictor
Linear
Regression
Reduced motion data
Reduced input data
Reduced motion data
Original motion data
Linear
Regression
Here, I am ready to talk about the kernel CCA-based regression. /First, we project training input and output data into each basis that are computed by kernel CCA. / We can obtain matrix A that links the reduced input to the reduced pose by linear regression. / Then we also obtain matrix B that recovers from the reduced pose to the original pose. /So, by concatenating these matrix, we can determine the predictor.

9. Motion synthesis Orientations of all joints Projection to the acquired
basis
Prediction matrix
Pre-operative gait
Post-operative gait
Here, I want to emphasize how to synthesize realistic motion. / We get the input data from /We then multiply it by prediction matrix. / Finally, we can get the all joints orientations of the body. The final pose is generated by joint mapping.

10. Result - method comparison

11. GCD Data normalization

12. Design X & Y
Pre-operative
Post-operative
C3D
C3D
GCD
GCD

13. Result - feature graph

14. Thank You
Q & A
Thank you for listening to my presentation.

15. Data & Feature
Many data has hundreds of variables with many irrelevant and redundant ones.
Feature is variables obtained by erasing redundant / noise variables from data.
보통 많은 케이스의 data는 수백 가지의 중복되거나 노이즈가 있는 variable들로 이루어져 있다.
Feature는 이러한 data로부터 중복되거나 노이즈가 있는 성분을 제거한 variable들로 이루어져 있다.

16. Advantages of feature selection
Alleviating the effect of the curse of dimensionality
Improve a learning algorithm’s prediction performance
Faster and more cost-effective
Providing a better understanding of the data
이렇게 만들어진 feature의 장점은 curse of dimensionality를 완화시킬 수 있다는 것인데 curse of dimensionality는…
따라서 data에서 추출된 feature는 원래 data보다 낮은 차원이고 따라서 curse of dimensionality를 피하여 잘 prediction할 수 있도록 해준다.
또한 계산시간이 빨라지고 data에 대한 이해도를 높여준다. 이 부분은 나중에 나오는 slide에서 설명해주겠다.

17. L1 regularization Effective feature selection method L1 norm:
- It is the sum of the absolute value of each component.

18. L1 regularization The L1 drives maximizes sparseness.
A new predicting post-operative gait can be estimated as matrix-vector multiplication.
- e.g.,)
L1 sparsity term

19. Result

20. L1 regularization
With the learned model , we can fully explain the features for each body joints.
- Features can be considered as the combination of the joint information corresponding non-zero terms in the row vector of the learned model.
- e.g. left knee position = 0.4 * left ankle position * pelvis position.

21. The problems
Can not explain the nonlinear relationship between training input and output.
Correspondence
Pre-operative
patient’s motion
Post-operative
patient’s motion

22. Regression process . . . . . . Pre-operative gait Regression
Pose to pose
. . .
Post-operative gait

23. Result – naïve, pose to pose

24. Minimizing prediction error
Motion to motion
- Considering a relation between remote poses
in temporal domain

25. Gait of cerebral palsy patient
이외에도 다리가 안쪽이나 바깥쪽으로 돌아가는 증상이 있을 수 있습니다.
출처:

26. Treatments
뇌성마비의 재활
보조기 활용
주사요법
신경외과적 수술
정형외과적 수술
약물치료

27. What is Cerebral palsy ? Cerebral palsy (뇌성마비) - 태아의 뇌에서 발생하는
비진행성 장애에 의한
활동 제한으로 동작과 자세에
영향을 미치는 영구적인 이상

28. Orthopedic surgery Distal Hamstring Lengthening (DHL)
Rectus Femoris Transfer (RFT)
Tendo Achilles Lengthening (TAL)
Femoral Derotation Osteotomy (FDO)
[Poses of cerebral palsy patients]

29. Related work
“Predicting outcomes of rectus femoris transfer surgery” [Reinbolt et al. 2009]
Select a set of preoperative gait features that distinguished between good and poor outcomes
“Evaluation of conventional selection criteria for lengthening for individuals with cerebral palsy” [Truong et al. 2011]

30. Related work [Chai and Hodgins 2005] [Slyper and Hodgins 2008]
기술적으로 관련된 연구로서 학습을 통한 캐릭터 애니메이션을 생성하는 연구입니다. 미리 녹화하여 저장된 사용자 입력과 그에 대응하는 사람 동작을 학습하여 새로운 사용자 입력에 대한 동작을 생성해내는 연구들이 있었다. 다양한 입력들로 마커, … 우리는 애니메이션 분야에 적용되었던 이러한 학습 기법을 토대로 수술 후 환자 보행을 예측하고자 합니다.
[Kim et al. 2012]
[Seol et al. 2013]

31. Motion data Number of patients Total seven joints DHL+RFT+TAL : 35
FDO+DHL+TAL+RFT : 33
Total seven joints
left foot
right foot
left femur
right femur
pelvis
left knee
right knee
3가지 혹은 4가지 수술을 받은 총 68명 환자의 수술 전과 수술 후 1, 2년 지난 환자의 동작 데이터를 학습에 이용하였다.

32. Naïve linear regression
Direct regression analysis between pre and post-operative gait
Minimize fitting error to obtain the predictor, .
Before I jump into the details of our prediction model, I want to talk about the Naïve linear regression. /The following equation minimizes the fitting errors between training input and output. We then obtain linear predictor, matrix A. / Although this formulation is very simple and easy to implement, there exists potential artifacts of the results because of large estimation errors. /This equation do not consider the data dependency between x and y. /So, the solution would come up with reducing the dimension of the training examples before entering the regression step.
Problems ? large prediction error

33. Result : motion to motion + naïve

34. Minimizing prediction error
Dimension reduction
Fully explains the nonlinear relationship between training input and output
- Nonlinear dimension reduction method

35. PCA PCA : maximum variance projection method. Data dependency ?
Variable X
Data dependency ?
Reduced Y
Reduced X
Projection
Variable Y
When we talk about dimensional reduction, there are many popular statistical methods. Maybe, PCA is the most familiar approach to choose. However, PCA might not be a good choice for regression purpose. As you can see here, we have two variables x & y that are training input and output, respectively. When we apply PCA to each variable, we can obtain the reduce the data by projection into the basis. We can represent the relationship of reduced x & y in 2D space. However, there is no guarantee pertaining to the data dependency of two variables in the reduced space. As a result, the prediction errors can not be minimized //
So, We now consider canonical correlation analysis, CCA. CCA finds the two different basis of each variable. The correlation between the two data sets is maximized in the reduced space. So it can reduce the estimation errors as associated with regression. In terms of approximating the original data, PCA is a good choice. PCA works by finding some basis that maximize variance of the data set.

36. Kernel CCA
CCA may not fully explain non-linear relationship between pre-operative and post-operative motion.
Non-linear CCA using the kernel trick method
Transform the data into high dimensional space
Substitute non-linear
mapping into CCA
However, one of the limitation of linear CCA is that it may not fully explain the non-linear relationship between input and output very well. So our key idea is that we map training data into the high-dimensional space, which is the feature space. By applying a function phi to training data, we can transform training data into the feature space. And we can reformulate CCA into Kernel CCA by using the kernel trick method.

37. Future work
Design training input & output with respect to the clinical context.
Feature selection
- Alleviating the effect of the curse of dimensionality
- Improve a prediction performance
- Faster and more cost-effective
- Providing a better understanding of the data
Mlml method. Feature selection 방법 정리.