1. QUANTITATIVE LITERACY PROGRAM RAJ BOPPANA MARY DIXSON KIM MASSARO GAIL PIZZOLA KIMBERLY WARD 2015 QLP Workshop

2. Agenda 9:30 – 9:50amIntroduction & Purpose of QLPRajendra Boppana 9:50 – 10:10 amQLCDS – Data Submission ProcessKim Massaro 10:10 – 10:20 amQ-course LogisticsKimberly Ward 10:20 – 10:40 amIncorporating QL into MAT 1043Jonathan Brucks 10:40 – 11:00 amIncorporating QL into WRC 1013/1023Gail Pizzola 11:00 – 11:30 amAnalysis of Q-course ResultsRajendra Boppana 11:30 – 12:45 amLunch Break 12:45 – 1:05pmIncorporating QL into CRJ 3013Rob Tillyer 1:05 – 1:25 pmIncorporating QL into KIN 3323Sakiko Oyama 1:30 – 2:30 pmDiscussion with the ProvostDr. Frederick 2:30 – 3:00 pmWrap Up and QuestionsQLP Team Location: JPL Faculty Center, Assembly Room

3. QLP Team Dr. Mary Dixson QLP Implementation/ Training Coordinator Kimberly Ward QLP Program Coordinator Dr. Gail Pizzola QLP Implementation/ Training Coordinator Dr. Rajendra Boppana Project Director of QLP Kim Massaro QLP Program Coordinator Prajan Pradhan QLP Data Specialist Robin Schulze QLP Coordinator & Analyst

4. Quantitative Literacy Program (QLP) Program Goals Develop quantitative skills in undergraduate students Implement effective teaching pedagogies and assessments to support the development of an exemplary quantitative scholarship program at the undergraduate level Provide the organizational framework and resources for an institutional transformation to graduate a quantitatively informed citizenry

5. Q-Course Current Course QLP Q-Course Learn Quantitative Skills Think critically Interpret and use data that naturally exist in the subject area Make informed decisions Makes the course more engaging Data + Q. Methods + Redesign

6. Student Participation in QLP Completion of one or more Q-courses is a graduation requirement Started with core courses, expanded to major- required upper division courses QLATAdvisingQ-coursesQLATGraduationStudents

7. Faculty Participation in QLP 167 faculty members, 150+ TAs/graders participated in QLP since Fall ‘11

8. QLP Timeline Year 1  First cohort of Q-faculty and students; 10 Q-courses  Program website is created  Every incoming freshman takes the entrance QLAT  Course data is collected Year 2  Developed online QLAT entrance exam  Workshop (QLW) is created to address core complete transfer students  Individual faculty and overall Q-course reports are developed  Surveys of students begin

9. QLP Timeline (contd.) Year 3  QLP maximized its enhancement of core courses  QLP invites upper division courses for redesign  Surveys to faculty, department chairs, and advisors begin  Exit QLAT is administered to compare to the baseline  Develops online version of QLW Workshop for transfer students  QLP awards first Faculty Excellence Award Year 4  Data Collection process is streamlined  8 upper division courses are enhanced with QL  Surveys to employers and alumni begin  QLP awards second Faculty Excellence Award

10. QLP Growth Year One (2011-12) Year Four (2014-15) 20 faculty100 faculty 10 Q-Courses27 Q-Courses 113 Sections556 Sections 6,845 enrollments26,599 enrollments

11. 2015-16 Q-course List ANT 2033 ANT 2043 ARC 4183 ARC 4283 BIO 1233 BIO 1404 COM 3073 CRJ 3013 ECO 2003 ECO 2013 ECO 2023 ENG 2413 ES 2013 HIS 2123 HIS 2133 KIN 3323 MAT 1043 MDS 4983 PHI 1043 POL 1013 POL 1113 SOC 1013 SOC 3323 SPE 3603 STA 1053 WRC 1013 WRC 1023 27 Q-courses 19 core, 8 upper division

12. Student Coverage: First-time, Full-time Students Completion of a Q-course by year by freshmen cohorts Each colored segment in a bar represents one year

13. Student Coverage: Transfer Students Completion of a Q-course by transfer student cohorts Each colored segment in a bar represents one year

14. Student Coverage: Graduating Students QLC: Completed at least one Q-course QLW: Completed a 3-hour workshop instead of a Q-course QLE: Exempted based on major; Q-course not completed

15. Student Enrollments in Q-Courses Fall 2015 enrollments are based on Aug 13 2015 data

16. Q-Course Performance Analysis (Fall 2014) 12 out of 19 (63%) of core level Q-Courses showed significant increase from pre to post-test  Out of those 12, 11 courses reported an average score greater than 70 on post-test. 5 out of 8 (63%) of upper-division Q-Courses showed significant increase from pre to post-test  Out of those 5, 4 courses reported an average score greater than 70 on post-test.

17. Q-Course Performance Analysis (Spring 2015) 13 out of 18 (72%) of core level Q-Courses showed significant increase from pre to post-test  Out of those 13, 10 courses reported an average score greater than 70 on post-test. 6 out of 8 (75%) of upper-division Q-Courses showed significant increase from pre to post-test  Out of those 6, all 6 courses reported an average score greater than 70 on post-test.

18. Presenter: Kim Massaro QLP Data Submission Process

19. Pre/Post Test Give the Pre-test before any Q-material is taught Grade the Express Question Bubble in the score for the Express question on the Parscore for each student Take Parscores to Testing Services Make sure to include a note: give permission to upload data to the QLP drive

20. How to Bubble Express Large Form: SUBJ Score 5 5 Small Form: Exam #

21. QLCDS Website qlcds.it.utsa.edu (open using Mozilla Firefox)  Log on with your abc123 and password Download your courses’ template  Course coordinators will create the templates at the beginning of the semester Upload the item level data

22. Course Coordinators Create pre-test template – Upload Pre-test document with SLO’s and taxonomy – Upload rubric for the Express Question with Answer Key – Upload a dummy file to generate SLO’s Create Homework template – Upload Q-assignment with SLO’s, taxonomy, and answers Create post-test template

23. Best Practices For Course Coordinators:  Create a course specific “primer” for faculty teaching  Ex: Materials handbook, Blackboard Learn shell, packet, etc…  Meet with new faculty prior to beginning of semester  Create QLCDS templates during first month of semester  When uploading materials, include SLOs, taxonomies, and correct answers clearly marked on document/rubric.

24. Best Practices (contd) For New Faculty teaching Q for first time:  Attend recommended trainings with the QLP team  Meet with course coordinator and other “Q” team members  Contact Testing Services for ParScore training (optional)  Visit QLP program website (http://qlp.utsa.edu) for more information on the program, workshop materials, video presentations, tutorials, and technical reports.http://qlp.utsa.edu

25. Best Practices (contd) For All Q-Faculty  Contact the Course Coordinator at beginning of each semester  Bookmark the following websites:  http://qlp.utsa.edu QLP program website http://qlp.utsa.edu  http://qlcds.it.utsa.edu Data Collection Website (very important!) http://qlcds.it.utsa.edu  http://qlp.utsa.edu/faculty (Resources include workshop materials, technical reports, and data collection tutorials) http://qlp.utsa.edu/faculty  https://medialibrary.utsa.edu/Brwose/Category/57 (video presentations from QLP staff and other Q-faculty) https://medialibrary.utsa.edu/Brwose/Category/57  Make sure TA/Grader attends QLP Training Workshop (8/28) or schedules one-on-one training  Verify that TA/Grader has all documents, rubrics, and understands the needs of the course.

26. Presenter: Kimberly Ward Q-Course Logistics

27. Beginning of the semester 1. Course Coordinator creates course template for pre-test, assignment, and post-test on QLCDS website 2. Faculty register pre/post-test with Parscore at Testing Services 3. Faculty meet with TA/Grader and provide documents and rubrics for grading Q materials. 4. Faculty give Blackboard gradebook access to TA/Grader (optional) 5. All Q-Faculty submit pre-test data by Wednesday Sept. 9 th  Must state “Give permission for Testing Services to send results to QLP”

28. Q-Course Logistics (contd.) Middle/End of the semester 1. Faculty or TA/Grader downloads “Homework” Excel template from QLCDS website 2. Faculty or TA/Grader enters student roster information and itemized student scores in Excel template columns. 3. Faculty or TA/Grader uploads the completed Excel template and indicates scoring criterion for each question. 4. “Q” Homework data is submitted to QLCDS by Friday Dec. 4 th 5. Post-test ParScore forms are dropped off at Testing Services by Friday Dec. 4 th 1. Unless on Final Exam, then due Tuesday Dec. 15 th (Grades Due Deadline)

29. Q-Course Logistics (contd) Should there be problems/missing data after submission, the QLP will email the faculty member to resolve the issue.  1 st Email – Individual Faculty  2 nd Email –Individual Faculty and Course Coordinator  3 rd Email—Individual Faculty, Course Coordinator, and Department Chair

30. Murphy’s Law and QLP When things go wrong/errors happen: TA/Grader  Assigned Faculty Faculty  Course Coordinator Course Coordinator  QLP Team Email the QLP team at qlp@utsa.eduqlp@utsa.edu NOT individual team member emails

31. QLP Data Submission Process & Due Dates Pre-test Data (via Parscore)  Due September 9 th (one week after census) Homework Data (for Upper-division courses only)  Due December 4 th (study days) Post-test Data (via Parscore)  Due December 4 th (study days)

32. Introduction to Mathematics Presenter: Jonathan Brucks Incorporating Quantitative Literacy into MAT 1043

33. Freshman Composition I and II Presenter: Gail Pizzola Incorporating Quantitative Literacy into WRC 1013/1023

34. Presenter: Rajendra Boppana Analysis of Q-course Results

35. Will resume at 12:45pm Break for Lunch

36. Agenda 12:45 – 1:05pm Incorporating QL into CRJ 3013 Rob Tillyer 1:05 – 1:25 pm Incorporating QL into KIN 3323 Sakiko Oyama 1:30 – 2:30 pmProvost’s DiscussionDr. Frederick 2:30 – 3:00 pmWrap Up and QuestionsQLP Team

37. Research Design and Analysis Presenter: Rob Tillyer Incorporating Quantitative Literacy into CRJ 3013

38. Biomechanics Presenter: Sakiko Oyama Incorporating Quantitative Literacy into KIN 3323

39. With Dr. John Frederick UTSA Provost and Vice President for Academic Affairs Round Table Discussion

40. Thank you for your participation.