Presentation is loading. Please wait.

Presentation is loading. Please wait.

Core Methods in Educational Data Mining

Published byLorin Kelley Modified over 6 years ago

Similar presentations

Presentation on theme: "Core Methods in Educational Data Mining"— Presentation transcript:

1 Core Methods in Educational Data Mining
EDUC545 Spring 2017

2 Diagnostic Metrics -- HW
Any questions about any metrics? Does anyone want to discuss any of the problems?

3 Diagnostic Metrics -- HW
When do you want to use fail-soft interventions?

4 Diagnostic Metrics -- HW
When do you not want to use fail-soft interventions?

5 Diagnostic Metrics -- HW
Q11 There’s been some debate on the forum as to what the right answer is Let’s compute this together in Excel Yes, you can do that

6 Textbook/Readings

7 Detector Confidence Any questions about detector confidence?

8 Detector Confidence What is a detector confidence value?

9 Detector Confidence What are the pluses and minuses of making sharp distinctions at 50% confidence?

10 Detector Confidence Is it any better to have two cut-offs?

11 Detector Confidence How would you determine where to place the two cut-offs?

12 Cost-Benefit Analysis
Why don’t more people do cost-benefit analysis of automated detectors?

13 Detector Confidence Is there any way around having intervention cut-offs somewhere?

14 Goodness Metrics

15 Exercise What is accuracy? Detector Academic Suspension
Detector No Academic Suspension Data Suspension 2 3 Data No Suspension 5 140

16 Exercise What is kappa? Detector Academic Suspension
Detector No Academic Suspension Data Suspension 2 3 Data No Suspension 5 140

17 Accuracy Why is it bad?

18 Kappa What are its pluses and minuses?

19 ROC Curve

20 Is this a good model or a bad model?

21 Is this a good model or a bad model?

22 Is this a good model or a bad model?

23 Is this a good model or a bad model?

24 Is this a good model or a bad model?

25 ROC Curve What are its pluses and minuses?

26 A’ What are its pluses and minuses?

27 Any questions about A’?

28 Precision and Recall Precision = TP TP + FP Recall = TP TP + FN

29 Precision and Recall What do they mean?

30 What do these mean? Precision = The probability that a data point classified as true is actually true Recall = The probability that a data point that is actually true is classified as true

31 Precision and Recall What are their pluses and minuses?

32 Correlation vs RMSE What is the difference between correlation and RMSE? What are their relative merits?

33 What does it mean? High correlation, low RMSE
Low correlation, high RMSE High correlation, high RMSE Low correlation, low RMSE

34 AIC/BIC vs Cross-Validation
AIC is asymptotically equivalent to LOOCV BIC is asymptotically equivalent to k-fold cv Why might you still want to use cross-validation instead of AIC/BIC? Why might you still want to use AIC/BIC instead of cross-validation?

35 AIC vs BIC Any comments or questions?

36 LOOCV vs k-fold CV Any comments or questions?

37 Other questions, comments, concerns about textbook?

38 Thoughts on the Knowles reading?

39 Thoughts on the Jeni reading?

40 Other questions or comments?

41 Next Class Wednesday, March 1 Feature Engineering
Baker, R.S. (2014) Big Data and Education. Ch. 3, V3, V4, V5 Sao Pedro, M., Baker, R.S.J.d., Gobert, J. (2012) Improving Construct Validity Yields Better Models of Systematic Inquiry, Even with Less Information. Proceedings of the 20th International Conference on User Modeling, Adaptation and Personalization (UMAP 2012), No HW due

42 The End

Download ppt "Core Methods in Educational Data Mining"

Similar presentations

Diagnostic Metrics Week 2 Video 3. Different Methods, Different Measures  Today we’ll continue our focus on classifiers  Later this week we’ll discuss.

Diagnostic Metrics Week 2 Video 3. Different Methods, Different Measures  Today we’ll continue our focus on classifiers  Later this week we’ll discuss.
...visualizing classifier performance Tobias Sing Dept. of Modeling & Simulation Novartis Pharma AG Joint work with Oliver Sander (MPI for Informatics,

...visualizing classifier performance Tobias Sing Dept. of Modeling & Simulation Novartis Pharma AG Joint work with Oliver Sander (MPI for Informatics,
Core Methods in Educational Data Mining HUDK4050 Fall 2014.

Core Methods in Educational Data Mining HUDK4050 Fall 2014.
Core Methods in Educational Data Mining HUDK4050 Fall 2014.

Core Methods in Educational Data Mining HUDK4050 Fall 2014.
The University of Texas at Austin, CS 395T, Spring 2008, Prof. William H. Press 1 Computational Statistics with Application to Bioinformatics Prof. William.

The University of Texas at Austin, CS 395T, Spring 2008, Prof. William H. Press 1 Computational Statistics with Application to Bioinformatics Prof. William.
Assessing and Comparing Classification Algorithms Introduction Resampling and Cross Validation Measuring Error Interval Estimation and Hypothesis Testing.

Assessing and Comparing Classification Algorithms Introduction Resampling and Cross Validation Measuring Error Interval Estimation and Hypothesis Testing.
Tutorial 2 LIU Tengfei 2/19/2009. Contents Introduction TP, FP, ROC Precision, recall Confusion matrix Other performance measures Resource.

Tutorial 2 LIU Tengfei 2/19/2009. Contents Introduction TP, FP, ROC Precision, recall Confusion matrix Other performance measures Resource.
CSCI 347 / CS 4206: Data Mining Module 06: Evaluation Topic 07: Cost-Sensitive Measures.

CSCI 347 / CS 4206: Data Mining Module 06: Evaluation Topic 07: Cost-Sensitive Measures.
Evaluating Classifiers

Evaluating Classifiers
Diagnostic Metrics, Part 1 Week 2 Video 2. Different Methods, Different Measures  Today we’ll focus on metrics for classifiers  Later this week we’ll.

Diagnostic Metrics, Part 1 Week 2 Video 2. Different Methods, Different Measures  Today we’ll focus on metrics for classifiers  Later this week we’ll.
Case Study – San Pedro Week 1, Video 6. Case Study of Classification  San Pedro, M.O.Z., Baker, R.S.J.d., Bowers, A.J., Heffernan, N.T. (2013) Predicting.

Case Study – San Pedro Week 1, Video 6. Case Study of Classification  San Pedro, M.O.Z., Baker, R.S.J.d., Bowers, A.J., Heffernan, N.T. (2013) Predicting.
Evaluation – next steps

Evaluation – next steps
Advanced Methods and Analysis for the Learning and Social Sciences PSY505 Spring term, 2012 February 13, 2012.

Advanced Methods and Analysis for the Learning and Social Sciences PSY505 Spring term, 2012 February 13, 2012.
Error estimation Data Mining II Year 2009-10 Lluís Belanche Alfredo Vellido.

Error estimation Data Mining II Year Lluís Belanche Alfredo Vellido.
Data Analysis 1 Mark Stamp. Topics  Experimental design o Training set, test set, n-fold cross validation, thresholding, imbalance, etc.  Accuracy o.

Data Analysis 1 Mark Stamp. Topics  Experimental design o Training set, test set, n-fold cross validation, thresholding, imbalance, etc.  Accuracy o.
Evaluating What’s Been Learned. Cross-Validation Foundation is a simple idea – “ holdout ” – holds out a certain amount for testing and uses rest for.

Evaluating What’s Been Learned. Cross-Validation Foundation is a simple idea – “ holdout ” – holds out a certain amount for testing and uses rest for.
Classification Performance Evaluation. How do you know that you have a good classifier? Is a feature contributing to overall performance? Is classifier.

Classification Performance Evaluation. How do you know that you have a good classifier? Is a feature contributing to overall performance? Is classifier.
Core Methods in Educational Data Mining HUDK4050 Fall 2014.

Core Methods in Educational Data Mining HUDK4050 Fall 2014.
Core Methods in Educational Data Mining HUDK4050 Fall 2014.

Core Methods in Educational Data Mining HUDK4050 Fall 2014.
Evaluating Results of Learning Blaž Zupan www.ailab.si/blaz/predavanja/uisp.

Evaluating Results of Learning Blaž Zupan

Similar presentations

Ads by Google