1. Evaluation Results of OP4L Learning Environment and Services in Slovenia Maribor, May 2012 Bar, Montenegro, 25 th – 26 th June Ines Kožuh, prof. dr. Matjaž Debevc

2. Research detailes The aim of our study: test OP4L learning environment and its services, assess students‘ relationship with collaborative learning evaluation: before testing OP4L learning environment after testing OP4L learning environment Company LOGO

3. Participants Computer Science students -1st year (Faculty of Electrical Engineering and Computer Science), volunteers signed consent form, 62 participants all male, age in years (Mdn = 20, Mean = 20.73, Min = 19, Max = 25), experience in programming. Company LOGO

4. Participants experience in programming (5 levels) Company LOGO

5. Procedure Three weeks in May 2012 Two classroom meetings: 1st meeting: before testing, instructions were given, questionnaires were filled in, 2nd meeting: after testing, questionnaires were filled in, The task: brainstorming, submission of proposed idea, programming, submission of the program, assessment of the program. Company LOGO

6. Measures Quantitative research metod (questionnaires), Evaluation before testing OP4L: A questionnaire on the habits of using Facebook, Facebook chat and Web resources, Evaluation after testing OP4L: A questionnaire on the habits of using Facebook, Facebook chat and Web resources (the same as before testing), A questionnaire on the experience in OP4L learning environment, A questionnaire on collaborative learning and using Facebook in favor of collaborative learning, A questionnaire on the usability of OP4L learning environment (SUS), Motivated strategies on learning questionnaire (MSLQ). Company LOGO

7. Measures Likert-type questions and open-ended questions questionnaires were put online. Company LOGO

8. Statistical analysis Descriptive statistics, definition of statistically significant relationships (Spearman‘s Rho coefficient), Kruskal-Wallis Test (differences among independent samples), SPSS 20.0. Company LOGO

9. Results Habits of using Facebook, chat and Web resources Comparison of the results before and after the study Company LOGO

10. Results Habits of using Facebook, chat and Web resources Company LOGO 52 8 2

11. Results Habits of using Facebook, chat and Web resources Company LOGO 18 32 10 2

12. Results Experience in OP4L learning environment Assessment of OP4L services: Recommended peers (idea vs. usage) Recommended readings (idea vs. usage) idea – from pedagogical and functional point of view usage – from pedagogical and functional point of view, technical operation IdeaUsage Nr of questions Cronbach‘s Alpha Nr of questions Cronbach‘s Alpha Recommended peers 5.7376.814 Recommended readings 4.5875.807

13. Results Experience in OP4L learning environment Company LOGO 3.7806 3.2231 3.8347 3.3742

14. Results Usability of OP4L learning environment (SUS) SUS score = 67.38 Company LOGO Source: A. Bangor, P. Kortum, J.A. Miller, The System Usability Scale (SUS): An Empirical Evaluation, International Journal of Human-Computer Interaction, 24, (2008), 6, pp. 574 ‒ 594.

15. Results Opinions on usability of OP4L learning environment Bad responsiveness of the system. Too many clicks were needed to be able to come to the task. Problems with submitting and assessing the tasks. Communication with others was too complicated. Company LOGO

16. Results Correlations between usability and OP4L services Company LOGO Correlations (1)(2)(3)(4)(5) (1) System usability Correlation Coefficient1,000.529 **.505 **.465 **.581 ** Sig. (2-tailed)..000 (2) Idea Recommended Peers Correlation Coefficient 1.000.620 **.449 **.401 ** Sig. (2-tailed)..000.001 (3) Usage Recommended Peers Correlation Coefficient 1,000.411 **.656 ** Sig. (2-tailed)..001.000 (4) Idea Recommended Readings Correlation Coefficient 1.000.504 ** Sig. (2-tailed)..000 (5) Usage Recommended Readings Correlation Coefficient 1,000 Sig. (2-tailed). **. Correlation is significant at the 0.01 level (2-tailed).

17. Results Collaborative learning and using FB in learning assessment of collaborative learning, assessment of using Facebook in learning, Company LOGO Number of questions Cronbach‘s Alpha Collaborative learning7.869 Facebook in learning4.868

18. Results Correlations among three variables Company LOGO Correlations (1)(2)(3) (1) Affinity for collaborative learning Correlation Coefficient1.000.452 **.352 ** Sig. (2-tailed)..000.006 (2) Affinity for using FB for solving complex problems Correlation Coefficient 1.000.404 ** Sig. (2-tailed)..001 (3) Frequency of using FB for solving complex problems Correlation Coefficient 1.000 Sig. (2-tailed). **. Correlation is significant at the 0.01 level (2-tailed).

19. Results Dispersion of data for two variables Company LOGO

20. Results Differences among groups in knowledge of programming Company LOGO Ranks Knowledge of programming NMean Rank Affinity for collaborative learning 2.004634.85 3.001421.11 4.00227.25 Affinity for using FB for solving complex problems 2.004633.38 3.001428.32 4.00210.50 Test Statistics a,b Affinity for collaborative learning Affinity for using FB for solving complex problems Chi-Square6.4803.691 df22 Asymp. Sig..039.158 a. Kruskal Wallis Test b. Grouping Variable: Knowledge of programming

21. Results MSLQ – Motivational strategies Company LOGO N = 58 students

22. Results MSLQ – Motivational strategies Company LOGO N = 58 students

23. Results MSLQ – Learning strategies Company LOGO N = 58 students

24. Results MSLQ – Learning strategies Company LOGO N = 58 students

25. Contact University of Maribor, Faculty of Electrical Engineering and Computer Science, Smetanova 17, 2000 Maribor www.feri.uni-mb.si Ines Kožuh e-mail: ines.kozuh@uni-mb.si prof. dr. Matjaž Debevc e-mail: matjaz.debevc@uni-mb.si Company LOGO