1. Database and Data Mining Tools for Electronic Teamwork Assessment Tool e-TAT Masters Project Oral Defense By Ravi Soni Graduate Student, Department of Computer Science San Francisco State University Project Advisors/Committee Members Dr. Dragutin Petkovic, Gary Thompson Dr. Marguerite Murphy Date: September 7 th 2010, CCLS Page 1

2. Outline  Problem Statement  Electronic Teamwork Assessment Tool (e-TAT)  e-TAT project  My Contributions to e-TAT  Use cases for Data Mining Tools  Requirements for DB, Access SW, DM Tools  Architecture  Implementation Issues  SE Methods Used  Testing  Conclusion  Future Work Page 2

3. Problem Statement Teamwork assessment is the measurement and analysis of team participation metrics during collaborative Software Development process Important for the instructors to know how well students apply, exhibit and subjectively experience the teamwork and soft skills during the development of a class project Difficult to measure Page 3

4. Background Teamwork assessment was done manually by using classical approach (assessed by exams) Was focused on “SE concepts, practices and theory” and not on how students apply them No method to automatically measure actual teamwork participation, usage of tools etc. Hence there was a need of an automated system, Electronic Teamwork Assessment Tool, e-TAT, to assess these soft skills Page 4

5. Electronic Teamwork Assessment Tool, e-TAT Combines survey tool, annotation and Data Mining Tools Developed by a team of four members at SFSU Designed to be consistent with different platforms Easy to use Database Schema to allow Data Mining Page 5

6. E-TAT Project Goals Collecting participation metrics from different platforms (Google groups, Google code) automatically Data Mining tools for analysis of the collected data Survey tool for conducting weekly surveys Storing instructor annotations for each group Page 5

7. My Contributions to e-TAT My Contributions to e-TAT Data Mining(DM) tools for analyzing stored data. DM tools include:  Total posts for groups  Correlation Graph  Monitor groups  Sanitize Database Design of the database schema Design and development of Data Access Software More info: [1] Page 5

8. Use cases for Data Mining Tools Instructor tries to assess groups in a class for grading In real time, the instructor uses e-TAT Data Mining Tools to track groups. Instructor wants to check all groups for the number of posts they have made till now Notes the group progress graph for the semester and grades individual groups Page 5

9. Requirements for Data Mining Tools Customizable graph to display the total posts for groups and correlation between post types Version 1 of “Total Posts for Groups” tool displayed graph for ‘forum’ posts (non-customizable) Post type, time frame and granularity options added during requirements gathering sessions Monitor groups daily, weekly, monthly. Send email to instructor if posts fall below limit Page 7

10. Requirements for Data Mining Tools Database sanitization to transform sensitive data in the database into sanitized data to be used for further analysis of data. Page 7

11. Requirements for Database Identification of data items to collect Identification of tables and relationships between them Integration of tables containing different types of data (User data, survey data, project data, instructors’ annotations) Page 7

12. Requirements for DB access software Providing layer at the database to store and retrieve data objects from the database Class to handle the database operations like connect, select, insert and update Page 8

13. Architecture and Design Page 9 Figure: Overall architecture of e-TAT and our contributions (shaded)

14. Implementation issues Page 9 No privileges for changing crontab using HTTP request To protect security of the server, we did not change system privileges for a particular user. We designed a temporary cron file (cron_replace.txt) containing list of schedules

15. Implementation issues Page 9 Figure: Privileges issue handled in “Monitor Groups” tool

16. Software Engineering Methods used Page 11 Agile Software Development SCRUM Meetings User Centered Design (UCD) for User Interface

17. Testing Methods Page 13 Black Box and White Box testing User Centered Approach for the UI Tested with original data available from ppm java.net website [3] Unoriginal data available using populate database scripts

18. Conclusions Page 14 Implemented Data Mining tools for e-TAT Designed and developed data access software for e-TAT Designed database schema for e-TAT

19. Future Work Page 15 Ability to further mine the sanitized data to find interesting patterns of group participations metrics over the semester Designed database schema to store data from different courses Analyze data integrated from different universities e.g. SFSU, Fulda university and Florida

20. Questions? Page 16

21. References [1] D. Petkovic, G. Thompson, R. Todtenhoefer, S. Huang, B. Levine, S. Parab, G. Singh, R. Soni, S. Shrestha : “Work in Progress – e-TAT: Online Tool for Teamwork and “Soft Skills” Assessment in Software Engineering Education”, submitted to Frontiers in Education FIE 2010 (undergoing final review) [2] Dragutin Petkovic, Rainer Todtenhoefer, Gary Thompson: “Assessment and Comparison of Local and Global SW Engineering Practices in a Classroom Setting”, ITiCSE’08, June 30-July02, Madrid, Spain [3] Software for Tracking and Analyzing Team Synergy [STATS], Software Engineering (Dr. Dragutin Petkovic and Gary Thompson), Fall 2008 SFSU, https://ppm-6.dev.java.net/ https://ppm-6.dev.java.net/ [4] M. Al-Yahya: “Using Wikis to Support teamwork Skills in Software Engineering Courses”, 22 nd Conf. on Software Engineering Education and Training, 2009 Page 25

22. References [5] Orit Hazzan, Y. D. (2008). Agile Software Engineering. Springer. [6] Google Chart Tools/ Image Charts. (2010). Retrieved from http://code.google.com/apis/chart/ http://code.google.com/apis/chart/ [7] Bulmer, M. (1979). Principles of Statistics. Oxford: Dover Publications, New York. [8] Pressman, R. (2005). Software Engineering: A Practitioner's Approach. Mc Graw Hill. Page 26

23. References [9] "Software Engineering 2004": Curriculum Guidelines for Undergraduate Degree Programs in Software Engineering, The joint Task Force on computing Curricula, IEEE Computer Society, Association for Computing Machinery, August 2004. [10] Aspray W., Mayadas F., Vardi M.Y., Editors: “Globalization and Offshoring of Software, A Report of the ACM Job Migration Task Force.” ACM 2006, http://www.acm.org/globalizationreport/ [11] Atlee, J.; leBlanc Jr., R.; Lethbridge, T.; Sobel, A.; Thompson, J.: “Reflections of Software Engineering 2004, the ACM/IEEE-CS Guidelines for Undergraduate Programs in Software Engineering”, P. Inverardi, M. Jazayeri (Eds.)., ICSE 2005 Educational Track, LNCS 4309, pp11-27, 2006, Springer Verlag, Berlin Heidelberg 2006 [12] Charette, R. N: “Why Software Fails.” IEEE Spectrum, September 2005, pp. 42. Page 27

24. References [13] Damian, Daniela; Hadwin, Allyson; and Al-Ani, Ban: “Instructional Design and Assessment Strategies for Teaching Global Software Development: A Framework.” Proceedings of the 28th international conference on Software Engineering (ICSE 2006) Shanghai, China, pp.: 685 – 690, 2006. [14] Sebern, M.: "The Software Development Laboratory: Incorporating Industrial Practice in an Academic Environment", Proc. 15th Conf. Software Engineering Education and Training (CSEE&T 2002) IEEE CS Press, Los Alamitos, Calif., 2002, pp. 118-127. [15] K. Garg, V. Varma: “Case Studies as Assessment Tools in Software Engineering Classrooms”, 22nd Conf. on Software Engineering Education and Training, 2009 [16] Host, M.: “Introducing Empirical Software Engineering Methods in Education.” Proc. Of the 15th Conf. on Software Engineering Education and Training (CSEET’02), 2002. [17] Huffman-Hayes, J.: “Energizing Software Engineering Education thought Real World Projects as Experimental Studies.” Proc. Of the 15th Conf. on Software Engineering Education and Training (CSEET’02), 2002. Page 28

25. Thank You! Page 17