1. Seeing the Forest and the Trees Seton Hall University Technology, Assessment and Strategic Planning Calvin Williams – Director of TLTC Janet Easterling – Institutional Research Associate Dr. Eric Fountain – Assessment Coordinator, IT Dr. Peter Ahr – Professor, Religious Studies

2. Copyright Peter Ahr, Janet Easterling, Eric Fountain, Calvin Williams, 2005. This work is the intellectual property of the authors. Permission is granted for this material to be shared for non- commercial, educational purposes, provided that this copyright statement appears on the reproduced materials and notice is given that the copying is by permission of the authors. To disseminate otherwise or to republish requires written permission from the authors.

3. A Process for Transformation: University Strategic Planning Integrating an Information Technology (IT) Long- Range Plan Implementing Quality Academic Technology Initiatives to meet Strategic Goals Developing Institutional and IT Assessment measures to enhance programming Building on Assessment and shared growth through Multi-Campus Assessment Questions and Answers: Major Presentation Themes

4. Seton Hall University Who Are We? We Are A Mid Sized, Private, Catholic Affiliated University in Central New Jersey Carnegie Classification: Research / Doctoral II Main Campus in South Orange, NJ –15 miles from New York City 10,000 Students –4,400 Full Time Undergraduates (40% Residential) 350 FT Faculty (450 FTE Faculty) FY’03 Annual Operating Budget Approx. $150 million –FY’02 G&E for South Orange Campus Approx. $115 million

5. Pace Of Change at Seton Hall University Before 1995: –Inadequate Computer Labs –Inadequate Local Networks/E-mail –Lack of Integration of Technology in Teaching –Disorganized Support/Allocation of Resources

6. Pace Of Change (Cont.) Now: Seton Hall University is doing IT “right” 2003 FORBES “America's Most Connected Campuses” - Ranked 16th 2000 EDUCAUSE Award (Honorable Mention) for Systemic Progress in T&L with Technology 1999 EDUCAUSE Award for Campus Networking Excellence Growing National Reputation of our Mobile Computing Initiative and other Teaching, Learning, and Technology Initiatives Alliance with IBM Corporation

7. A Vehicle for Change: University Strategic Planning Seton Hall University has accomplished heightened recognition among top-tier Catholic Universities nationally The strategic goals include a critical focus on the intellectual, personal and spiritual development of all students Students experience a rigorous, value-centered and learner-centered technologically enhanced environment

8. Advancing Strategic Goals with Technology It was recommended that a strategic agenda include an Information Technology Strategic Planning process to build an Long-Range IT Plan

9. Advancing Strategic Goals with Technology To advance the strategic goals, a strong focus was placed on communication, teaching and learning as well as support services for students using technology.

10. Core Curriculum Reform Faculty Senate Committee elected in 2000 Request of the President Rounds of meetings with all faculty Analysis of possible models Focus on skills and literacies rather than content Emphasis on assessment and accountability

11. Core Curriculum Proposal Three courses developing institutional mission and student self-learning Systematic development of fundamental skills Integration of key literacies Electronic portfolios for all students Emphasis on assessment of outcomes of education

12. Core skills Critical thinking Writing Oral presentation Information fluency Specific metrics developed for integration into at least 50% of a student’s courses Conscious development of these habits of mind as part of coursework

13. Fundamental Literacies Religious literacy Ethical literacy Quantitative literacy Scientific literacy Aesthetic literacy Historical/global literacy All students must take at least four courses which develop these literacies

14. E-portfolios Database of courses certified as fulfilling the Core requirements Ongoing checklist of student’s fulfilling Core requirements for advising Depository of objects attesting to student’s skill development Opportunity for student reflection on the learning process

15. Assessment via e-portfolios Assessment of student learning Faculty accountability for quality of work products Public accountability of effectiveness of programs Record for evaluation of success of Core Accountability of all parties System of institutional assessment

16. Value of technology to Core reform Based on mobile computing Builds on existing initiatives Uses our success in information technology as a model for further innovation E-portfolios as public record of student development Assessment of student learning as required by Middle States

17. Division of Information Technology Assessment History Mobile Computing Assessment Metrics Program

18. Internal Focus Started in PC Support Performance Measures Well Established Facilitated by New Ticketing System Focus on Organizing and Displaying Information and Cost Metrics for ’04-’05

19. University Level –Mobile Computing Assessment Team –Teaching, Learning, and Technology Roundtable –Institutional Research –Faculty Senate Program Level –ePortfolio Course Level –Large Course Redesign Multi-Campus Mobile Computing Assessment Strategies

20. Narrowing the Question Fantasy: a + b = c a = teaching, b = technology, c = learning Reality: a + b + c + d + e + f… discipline, teaching style, technology proficiency, fit of application to task, desired learning objectives...

21. Mobile Computing Assessment Team Cross-Campus, Cross-Discipline Provost’s Office Institutional Research Faculty Instructional Design Multiple Projects Mobile Computing Assessment Survey Extant Data Analysis Large-Course Redesign Multi-Campus

22. Mobile Computing Survey R1. What is the effect of the Mobile Computing Program on student satisfaction? R2. What is the effect of the Mobile Computing Program on technology use? R3. What is the effect of the Mobile Computing Program on desired educational outcomes? Research Questions

23. Technology Use Seton Hall Students 1998-2000

24. Technology Use – Students 1998-2000 2002 2003 6+ hours/week 6+ hours/week Daily,Weekly Communicating 44% 49% 48%,35% Researching 34% 34% 26%,47% Blackboard 14%46%,18% (43% 2+)

25. Satisfaction with Technology – Students Satisfied or Highly Satisfied 1998-2000 2002 2003 Mobile Courses 56% 76% 93% Pre-Loaded Software 73% 93% Laptop Computer 68% 75% 86%

26. Impact of Technology on Learning Student Perceptions 1998-2000

27. Impact of Technology on Learning Student Perceptions 1998-2000 2002 2003 Impact Positive or Very PositiveOften+ Contact with professors 78% 95% 61v74% Working with other students 67% 85%50v56% Discuss/write/think in depth 61% 80%64v64%

28. NSSE benchmarks for Effective Educational Practices Level of Academic Challenge Active & Collaborative Learning Enriching Educational Experiences Supportive Campus Environment Student Faculty Interaction

29. What NSSE Measures Student exposure to and participation in educational practices that promote engagement and student learning –Interaction with faculty in & out of the classroom –Active & collaborative learning –High expectations and academic challenge –Enriching educational experiences Student perception of a supportive campus environment helping them to succeed

30. NSSE Benchmarks and SHU SHU strengths, according to NSSE ?Connections to Technology? Active and Collaborative Learning Interactions with Seton Hall Faculty

31. NSSE Benchmark Active/Collaborative Learning In each NSSE year (’00, ’01, ’02, ‘03): SHU’s average Freshman score exceeded that of 80% of institutions in the same Carnegie classification category. SHU’s average Senior score exceeded that of 40-60%

32. NSSE Questions - Active & Collaborative Learning

33. NSSE 2003 - Active & Collaborative Learning Seton Hall Freshmen vs Peers & National

34. NSSE Benchmark Faculty-Student Interaction In each NSSE year (’00, ’01, ’02, ‘03): SHU’s average scores exceeded that of 80% - 90% of institutions in the same Carnegie classification category. For Freshmen*, SHU exceeded 90-100%! *In 2000, NSSE cited Seton Hall for this exemplary result

35. NSSE Questions - Faculty Student Interaction

36. NSSE 2003 – Faculty Student Interaction Seton Hall Freshmen vs Peers & National

37. Effective Learning Environments … Connections to Technology? Are positive NSSE outcomes evidence that SHU investments in IT help build SHU engaging educational environments? NSSE experimental Technology Qs (2003)

38. NSSE Technology Qs Technology Use SHU StudentsSHU Carnegie Peers Time online for academic - 6+ hours / week 55% 2003 SHU FR40% Doct-Intensives FR 51% 2003 SHU SR44% Doct-Intensives SR Time online for any reason - 6+ hours / week 88% 2003 SHU FR77% Doct-Intensives FR 81% 2003 SHU SR72% Doct-Intensives SR

39. NSSE Technology Qs Technology Use & Active Learning SHU StudentsSHU Carnegie Peers How often required to use IT - % Very Often 59% 2003 SHU FR49% Doct-Intensives FR 61% 2003 SHU SR56% Doct-Intensives SR Used T to make presentations - % Very Often 47% 2003 SHU FR34% Doct-Intensives FR 56% 2003 SHU SR53% Doct-Intensives SR

40. NSSE Technology Qs Technology Use & Faculty Contact SHU StudentsSHU Carnegie Peers Email ask instructor to clarify assign - % Very Often 51% 2003 SHU FR30% Doct-Intensives FR 57% 2003 SHU SR39% Doct-Intensives SR Email to express ideas to instructor - % Very Often 22% 2003 SHU FR13% Doct-Intensives FR 28% 2003 SHU SR16% Doct-Intensives SR

41. NSSE Technology Qs Technology Use & Active/Collaborative Learning SHU StudentsSHU Carnegie Peers Work in teams in class using IT - % Very Often 27 % 2003 SHU FR14 % Doct-Intensives FR 25 % 2003 SHU SR19 % Doct-Intensives SR Communicate with classmates on-line to complete academic work - % Very Often 51% 2003 SHU FR27 % Doct-Intensives FR 49 % 2003 SHU SR39 % Doct-Intensives SR

42. NSSE Technology Qs Technology Use & Active Learning / Reflection SHU StudentsSHU Carnegie Peers Gain new insights into course material from on- line discussions - % Quite a bit+ 43% 2003 SHU FR37% Doct-Intensives FR 31% 2003 SHU SR32% Doct-Intensives SR Express ideas to instructor by email not comfortable saying in class - % Very Often 22% 2003 SHU FR13% Doct-Intensives FR 28% 2003 SHU SR16% Doct-Intensives SR

43. Conclusion/suggestion post review of NSSE benchmarks & NSSE Technology Qs SHU NSSE & technology results are consistent and together suggest that SHU investments in IT are helping build educational environments at SHU that are increasingly engaging, challenging, learning community oriented, reflective and “active”.

44. Mobile Computing Conclusions Attracted by the availability of technology at the University and the infusion of technology in the curriculum Satisfied with the Mobile Computing Program as well as with the laptop and support services Making good use of the technology available Perceiving a substantive positive impact of Seton Hall University’s Mobile Computing Program on the learning environment. The Seton Hall University student is generally:

45. MCAT – Future Plans Expand Faculty Participation Organize and Publicize Results Approach Question of Classroom Use FlashLight Comparison Studies

46. Teaching, Learning, and Technology Roundtable Cross-Campus Representation Assessment Subcommittee –Studying faculty adoption/refusal rates and patterns.

47. Other Campus-Wide Efforts Faculty Senate – IT Subcommittee Outcomes Assessment Accreditation Self-Study e-Portfolios

48. Seton Hall University Teaching, Learning and Technology Center Walsh Library, South Orange, NJ 07079 973-275-2929 Thank You! Calvin Williams – Director of TLTC Janet Easterling – Institutional Research Associate Dr. Eric Fountain – Assessment Coordinator, IT Dr. Peter Ahr – Professor, Religious Studies