1. Tracking and Analyzing Student Success Data Part 3

2. State/Territory Area (km 2 ) Afghanistan 645,807606,467 Albania 28,74825,643 Algeria 2,381,7412,556,276 American Samoa 197205 Andorra 464577 Anguilla 9688 Antigua & Barbuda 442498 Argentina 2,777,4092,537,555 Armenia 29,74325,859 Aruba 193210 Australia 7,682,5577,365,389 Austria 83,85881,434 Azerbaijan 86,53091,752 Bahamas 13,9629,125 Bahrain 694725 Bangladesh 142,61598,722 Barbados 431517 Belgium 30,51825,417 Belize 22,96520,648 Benin 112,620108,768 ……… Which column lists the correct areas?

3. A Bit of History Late 1800s: Astronomer Simon Newcomb noticed that the early pages of log table books were more grubby than the later pages  Users were looking-up numbers that started with digit 1 more often than numbers starting with, say, digit 5. If the leading (first) digit is d, then the frequency of occurrence (probability) of the leading digit is Log 10 (1 + 1/d)

4. If the leading (first) digit is d, then the frequency of occurrence (probability) of the leading digit is Log 10 (1 + 1/d) Leading digit (d) 123456789 Probability of occurrence 30%18%12%10%8%7%6%5%< 5% NumberLeading (first) digit 3503 420574 0.646

5. A Bit of History (cont’d) Late 1930s: Physicist Frank Benford rediscovered Newcomb’s formula: Log 10 (1+1/d)  Benford’s Law Source: The Law of Anomalous Numbers, F. Benford, Proceedings of the American Philosophical Society, Vol. 78, 1938, pp: 551-572.

7. State/Territory Area (km 2 ) Afghanistan 645,807606,467 Albania 28,74825,643 Algeria 2,381,7412,556,276 American Samoa 197205 Andorra 464577 Anguilla 9688 Antigua & Barbuda 442498 Argentina 2,777,4092,537,555 Armenia 29,74325,859 Aruba 193210 Australia 7,682,5577,365,389 Austria 83,85881,434 Azerbaijan 86,53091,752 Bahamas 13,9629,125 Bahrain 694725 Bangladesh 142,61598,722 Barbados 431517 Belgium 30,51825,417 Belize 22,96520,648 Benin 112,620108,768 ………

8. State/Territory Area (km 2 ) Afghanistan 645,807606,467 Albania 28,74825,643 Algeria 2,381,7412,556,276 American Samoa 197205 Andorra 464577 Anguilla 9688 Antigua & Barbuda 442498 Argentina 2,777,4092,537,555 Armenia 29,74325,859 Aruba 193210 Australia 7,682,5577,365,389 Austria 83,85881,434 Azerbaijan 86,53091,752 Bahamas 13,9629,125 Bahrain 694725 Bangladesh 142,61598,722 Barbados 431517 Belgium 30,51825,417 Belize 22,96520,648 Benin 112,620108,768 ……… Correct List Source: Improvised from R. M. Fewster, The American Statistician, Feb. 2009, Vol. 63, No. 1.

9. Benford’s Law in Mastering TM Rasil Warnakulasooriya Pearson, Boston

10. A Quick Demo of Mastering TM masteringphysics.com Thanks to Claire Masson for the Demo slides

11. A Tutorial Problem in MasteringPhysics ®

12. Student enters 0 to Part A

13. Students can open Hints

14. Declarative Hint Procedural Hint

15. Feedback within Hints

16. Student obtains the correct answers

17. Time to Completion Data StudentItemTime interval (sec.) [finished time-start time] First (leading) digit S1Tutorial X3503 S1EOC Y10481 S3EOC Y676 S4Tutorial Z421794 …………

18. MasteringPhysics ® time data follow Benford’s Law closely

19. MasteringChemistry ® time data follow Benford’s Law closely (better than in physics)

20. But what can we do with it?

21. What can the deviations from the Benford’s Law tell us?

22. Deviation from Benford’s Law less deviationmore deviation

23. Final Exam Score vs. Deviation from Benford’s Law (Chemistry) Thanks to professors Randall Hall & Leslie Butler for providing the exam score data

24. Final Exam Score vs. Deviation from Benford’s Law For every 1/5 th of deviation from Benford’s Law, the exam score is decreased by 1/2 standard deviation

25. To what degree is the average time correlated with the final exam scores? -0.03 (about 7 times less than the correlation found by using only the first digits; almost no correlation)

26. The correlation (-0.2) is small but surprisingly good (?) * Typical correlations encountered in educational research ~ 0.3 * High school GPA & SAT ® -math correlated at 0.23  * SAT ® -math and first-year college GPA correlated at 0.26   Source: College Board Research Report No 2008-5: Validity of the SAT ® for Predicting First-Year College Grade Point Average, Jennifer L. Kobrin, et al.

27. Can we identify at-risk students?

28. Using average time: 0% of the students can be flagged Students who sat for the final exam Students who did not sit for the final exam

29. Using Benford’s Law: ~19% of the students can be flagged Students who sat for the final exam Students who did not sit for the final exam

30. * In time to completion data the first digit (or deviations from the Benford’s Law) carries more actionable information than all of the digits combined! Summary * Possibility of weighting the deviations from Benford's Law to identify at-risk students so that instructors can intervene effectively * Possibility of weighting the deviations from Benford’s Law in deciding on specific study plans/personalized learning resources within Mastering

32. State/Territory Area (miles 2 ) Afghanistan 249,347234,158 Albania 11,1009,901 Algeria 919,595986,983 American Samoa 7679 Andorra 179223 Anguilla 3734 Antigua & Barbuda 171192 Argentina 1,072,363979,755 Armenia 11,4849,984 Aruba 7581 Australia 2,966,2502,843,791 Austria 32,37831,442 Azerbaijan 33,40935,426 Bahamas 5,3913,523 Bahrain 268280 Bangladesh 55,06438,117 Barbados 166200 Belgium 11,7839,814 Belize 8,8677,972 Benin 43,48341,996 ………