1. Michael Jennings ACSME 2014 Michael Jennings (UQ), Deb King (Melbourne), Adam Bridgeman (Sydney), Liz Johnson (Latrobe/Deakin) Not just another diagnostic test!

2. Michael Jennings Today’s talk… …is brought to you by the Office of Learning and Teaching of the Australian Government.

3. Michael Jennings B.Engineering Numbers Yr’91’01’02’03’04’05’06 No.400600560530520570750 Yr’07’09’10’11‘12’13’14 No.850970100011001200 Bachelor of Engineering first-year numbers (UQ enrolment data) Prereqs : 1991 - English, Maths 1 (Maths B), Maths 2 (Maths C), Physics, Chemistry 1995 onwards - English, Intermediate Maths (B), Physics OR Chemistry Approximately 70% of students come in with both Intermediate & Advanced Mathematics (B & C)

4. Michael Jennings Diagnostic Test Leo Howard Test 1972-1995 Paper test – 2007 Online – 2009 to present History

5. Michael Jennings Current test – 2009 to present UQ T&L grant – cross-discipline team of Engineering & Science (inc. Maths) Background questions Learning approaches F-10 Maths & Senior Intermediate Maths Physics Chemistry

6. Michael Jennings Current test – OLT grant Led by USQ, involving UQ, UNE, UTS, and UNewcastle http://getset.ceit.uq.edu.au/ 20 maths questions –18 MCQs, 2 enter the answer – Can’t remember & never seen before options 12 physics and 12 chemistry questions

7. Michael Jennings Aim of the test Is NOT to tell students how much they don’t know / forgotten Is to gauge current mathematical / science understanding Is to tell students what maths/science is used in their courses so they can choose the appropriate courses Is to identify at-risk students

8. Michael Jennings Results Consistent over the years Students who have done both Intermediate and Advanced Mathematics (Maths B & C) at school do better Senior topics not done as well as junior ones Differentiation and integration Correlation between test results, ATAR and success in first-year engineering

9. Michael Jennings Students’ thoughts Benefit from doing quiz? % response It gave me insight into the knowledge I need for 1st year60 It flagged some things I have forgotten and need to revise82 I feel this is the beginning of my learning journey at uni29 It made me feel that my needs were being assessed29 No, it achieved none of the above2

10. Michael Jennings Are you glad you did it? % Yes, I have a better idea of what I know and don't know73 Yes, I have a better understanding of 1st yr knowledge expectations62 Yes, I feel more confident26 I’m not glad I did it at all4 What actions will you take as a result? % Revise my high school notes59 Find a relevant text book40 Ask someone who knows for help25 Do nothing as I'll pick it up during semester17 Do nothing because I did well8

11. Michael Jennings What did you dislike about doing the quiz? % I haven't disliked doing it at all38 I forgot a lot of what I knew during the holidays43 It has scared me11 It was stressful6 It didn't help2 It was time consuming and boring8 I was still in a holiday mood35 Kavanagh, L., O’Moore, L, & Samuelowicz, K. (2009). Characterising the first year cohort knowledge. Proceedings of the AAEE 2009 conference. Burton, L., Dowling, D., Kavanagh, L., O’Moore, L, & Wilkes, J. (2012). Examining First Year Students’ Preparedness for Studying Engineering. Proceedings of the AAEE 2012 conference.

12. Michael Jennings Question NoConcept Correct Answers % Q6Add two numerical fractions97.54 Q7Add two algebraic fractions77.05 Q8Solve an equation92.87 Q9Expand a factorised quadratic89.47 Q10Factorise a quadratic91.95 Q11Solve a quadratic83.76 Q12Solving simultaneous equations x=70.04 y=71.98 Q13Simplify an expression with powers76.46 Q14Understanding sine, cos, and tan84.89 Q15Straight line equations What is the slope of the line?88.99 What is the y intercept?78.52 Q16Simple derivative92.72 Q17Optimising a function77.67 Q18Visualising a derivative65.22 Q19Adding different base logarithms64.1 Q20Derivatives - Product rule49.38 Q21Derivatives - Chain rule73.45 Q22Simple integral89.66 Q23Definite integral64.81 Q24Calculate the area under graph76.6 Q25Composition of functions64.62 2013

13. Michael Jennings Feedback matrixmatrix http://getset.ceit.uq.edu.au/

14. Michael Jennings OLT Extension Grant 2014-2015 Melbourne, Sydney, Latrobe science, mathematics and business programmes UQ Science Macquarie Southern Cross, Charles Sturt, Griffith, UWS

15. Michael Jennings What maths is in your course(s)? What do you want to know? –Procedural vs conceptual understanding. With what do you think students struggle? When are you going to do the test? What are you going to do with the info from the test? QS in Science – www.qsinscience.com.auwww.qsinscience.com.au

16. Michael Jennings IT details Drupal 7 Blackboard + other programs

17. Michael Jennings Acknowledgements The 2009 online diagnostic test was funded by a The University of Queensland Teaching and Learning grant. The project team was A/Prof Lydia Kavanagh (Faculty of Engineering, Architecture and Information Technology), Dr Liza O’Moore (School of Civil Engineering), Professor Peter Halley (School of Chemical Engineering), Professor Paul Lant (School of Chemical Engineering), Professor Peter Adams (Faculty of Science), Michael Jennings (School of Mathematics and Physics), Professor Lawrie Gahan (School of Chemistry & Molecular Biosciences) and Dr Anton Raynor (School of Mathematics and Physics),. The OLT funded project involved USQ, UQ, UNE, UTS and UNewcastle. http://www.olt.gov.au/project-get-set-success-using- online-self-assessments-motivate-first-year-engineering-students- engaghttp://www.olt.gov.au/project-get-set-success-using- online-self-assessments-motivate-first-year-engineering-students- engag The feedback matrix was designed by A/Prof Lydia Kavanagh (Faculty of Engineering, Architecture and Information Technology), Dr Liza O’Moore (School of Civil Engineering).

18. Michael Jennings Burton, Lorelle, Dowling, David, Kavanagh, Lydia, O'Moore, Liza & Wilkes, Janelle (2012). Examining first year students' preparedness for studying engineering. In: Llewellyn Mann and Scott Daniel, AAEE 2012 Conference Proceedings. 23rd Annual Conference of the Australasian Association of Engineering Education (AAEE 2012), Melbourne, Australia, (1020-1028). 3 - 5 December 2012. Jennings, M. (2011). The transition from high school to university: The University of Queensland perspective. In: John Hannah, Mike Thomas and Louise Sheryn, Te Ara Mokoroa: The Long Abiding Path of Knowledge - Proceedings of Volcanic Delta 2011. Volcanic Delta 2011: The Eighth Southern Hemisphere Conference on Teaching and Learning Undergraduate Mathematics and Statistics, Rotorua, New Zealand, (139-149). 27 November - 2 December 2011. Kavanagh, L., O’Moore, L., & Samuelowicz, K. (2009). Characterising the first year cohort knowledge. 2009 Conference Proceedings. Annual Conference of the Australasian Association of Engineering Education (AAEE 2009). References