ASCP Workshop 04June04 LA076 SDSU Evaluation & Assessment in the Undergraduate Curriculum Assessment as Collaboration Kris Stewart NPACI Education Center on Computational Science and Engineering San Diego State University Professor, Computer Science

ASCP Workshop 04June04 LA076 SDSU What is Computational Science? Science Discipline Physics, Chemistry, Biology, etc. Computer Science Hardware/Software Applied Mathematics Numerical Analysis, Modeling, Simulation Computational Science Teamwork and Collaboration

ASCP Workshop 04June04 LA076 SDSU What is Computational Science? Recognized as separate discipline E.g. Society for Industrial and Applied Mathematics and Osman Yassar, IEEE Computing in Science & Engineering Should CSE be a separate major? (Computational Science [no, too general] or Computational Physics [yes, natural extension]) Should it be a separate department? (How to support interdisciplinarity?)

ASCP Workshop 04June04 LA076 SDSU Numerical Analyst* led to Supercomputing and Undergraduate Education (SUE**) led to Supercomputing Teacher Enhancement Program (STEP***) led to Education Center on Computational Science & Engineering (EC/CSE) Kris’ Faculty Background (Kris Stewart, Director, San Diego State University, California State University System) *MS/CS SDSU 1979, JPL 1981, PhD UNM 1987, SDSU 1986 ** UCES (DoEnergy 1994) *** Smithsonian Research Collection (1996)

ASCP Workshop 04June04 LA076 SDSU UCES Paradigm (thanks Tom Marchioro and the “crew”, 1994?) Previous exposure to “assessment”

ASCP Workshop 04June04 LA076 SDSU 1998/99 Assessment by LEAD Background Workshop in Wisconsin April 1997 to learn about assessment and make it real NPACI starts 1 October97 EC/CSE requested assessment for 1998 Assessment as a Collaboration Preparation for SDSU Campus Visit by Baine & Julie Discussions at SC98- SuperComputing Fall98 Orlando to SDSU faculty gather attitudes

ASCP Workshop 04June04 LA076 SDSU Grand Challenges for HPC (Stewart & Zaslavsky, SC98) 1.Faculty system of rewards does not encourage teaching innovations 2.Lack of awareness of HPC technologies already used in research or teaching for different fields 3.Faculty & students unaware of benefits and accomplishments of HPC 4.HPC technologies considered too complex/inaccessible for undergraduate instruction 5.Sequential HPC-related curricula is absent 6.Curricula using very large data sets not widely available 7.Adjust to different learning styles when material is complex 8.Variety of platforms/software leads to fragmented curricula 9.School administration/support staff not ready for HPC 10.Specs of computers and networks below user expectations We had been thinking about this (based on April 97 LEAD Workshop in WI)

ASCP Workshop 04June04 LA076 SDSU Building the Community of Faculty These challenges are people-centric, not technology-centric and of interest to the community Systemic Change requires understanding the system and working within it Empower teachers (find the time), recognition (from chair/dean), support (student assistants)

ASCP Workshop 04June04 LA076 SDSU Interviews on SDSU Campus LEAD applying info from surveys (Spr 99) Faculty skepticism: Convincing evidence that computer-based tools enhance teaching process? Knowledge of modern computational methods and availability? Incentive from department and insufficient tech support? LEAD Interviews with V.P., Deans, Chairs Faculty Fellows program identified as a target OUTCOMES: our local infrastructure at SDSU took us more seriously survey instruments refined SWB value recognition

ASCP Workshop 04June04 LA076 SDSU Assessment not just requirement Rather, found to be vital tool to assist in clarifying student and faculty needs improve prioritization skills validation of focus on human factors to integrate HPC (modeling & visualization) into undergrad curriculum

ASCP Workshop 04June04 LA076 SDSU California Education Infrastructure K12 Education (standards based, performance based) Community Colleges (Freshman/Sophomore) –Vocational –University preparation California State University System (23 campuses) University of California (9 campuses, Merced in 2004) Independents (Stanford U., CalTech, U. Southern California)

ASCP Workshop 04June04 LA076 SDSU California Education Infrastructure (testbed for change) IMPAC Intersegmental Major Preparation Articulated Curriculum Project Community Colleges and Four Year Universities Charter Schools (Preuss School) preuss.ucsd.edu Dr. Rozeanne Steckler and Dr. Mike Bailey “Fostering Scientific Curiosity in All Children” San Diego Supercomputer Center

ASCP Workshop 04June04 LA076 SDSU Involving University Faculty Infrastructure for Change NPACI/SDSU Faculty Fellows Local Support from College Deans and Department Chairs (participation buy-in and faculty recognition) SDSU Academic Advisors (across disciplines) Professional Meeting (SC2001, SIAM, ACM, SIGCSE, your suggestions?)

ASCP Workshop 04June04 LA076 SDSU Grand Challenges for HPC (Stewart & Zaslavsky, SC98) 1.Faculty system of rewards does not encourage teaching innovations 2.Lack of awareness of HPC technologies already used in research or teaching for different fields 3.Faculty & students unaware of benefits and accomplishments of HPC 4.HPC technologies considered too complex/inaccessible for undergraduate instruction 5.Sequential HPC-related curricula is absent 6.Curricula using very large data sets not widely available 7.Adjust to different learning styles when material is complex 8.Variety of platforms/software leads to fragmented curricula 9.School administration/support staff not ready for HPC 10.Specs of computers and networks below user expectations

ASCP Workshop 04June04 LA076 SDSU Building the Community of Faculty These challenges are people-centric, not technology-centric and of interest to the EPSCOR community Systemic Change requires understanding the system and working within it Empower teachers (find the time), recognition (from chair/dean), support (student assistants)

ASCP Workshop 04June04 LA076 SDSU Faculty Fellows Fall 01 Synergy among themselves and with their chairs and deans People, Time, Support, Recognition …

ASCP Workshop 04June04 LA076 SDSU NSF/EHR National Science Foundation/Education and Human Resources Directorate RED/EVAL/handbook/ handbook.htm LEAD Assessment and Evaluation 1998 Formative for the EC/CSE ~lead/pages/produts/eot- paci.pdf

ASCP Workshop 04June04 LA076 SDSU EC/CSE involve CSU Faculty Workshops Introduce Computational Science Host Biology Workbench (BioQuest) Support at NPACI AHM02 Extend Faculty Fellows

ASCP Workshop 04June04 LA076 SDSU Contact Us Kris Stewart, Director Jeff Sale Kirsten Barber

ASCP Workshop 04June04 LA076 SDSU The Ed Center Project and its Evaluation Goal: incorporate computational science techniques in undergraduate curricula Differences from STEP: target audience, institutional arrangements Methods and Strategies: Faculty Fellows Workshops, presentations, in-house projects, trying out new approaches in own teaching, tools development Computational Science Olympics (curriculum from the bottom- up!!) Surveys, continuous assessment of faculty involvement and learning outcomes Evaluated by the LEAD Center in

ASCP Workshop 04June04 LA076 SDSU Introducing the EC/CSE The mission of the Ed Center on Computational Science and Engineering? Who are the people involved? Some of our projects: Some of our activities: Some resources:

ASCP Workshop 04June04 LA076 SDSU Undergraduate Faculty: A Tough Target Group Obstacles: lack of time, tenure and review considerations, lack of awareness about available technologies Undergraduate faculty: –¾ have used WWW often or sometimes (1997), but not in the classroom (only 18% ) –The gap between those NEVER using computers in the classroom, and those using them OFTEN, is the largest for untenured faculty, increasing towards tenure review –Only 12% of surveyed faculty saw themselves as having a use for HPC applications in courses (higher for Sciences and Engineering) –11% of faculty have students working with computer models OFTEN

ASCP Workshop 04June04 LA076 SDSU Using computers in the classroom versus number of years as a faculty member (1997 Faculty Survey)

ASCP Workshop 04June04 LA076 SDSU Primarily graduate facultyPrimarily undergraduate faculty College of Arts & LettersCollege of Sciences 1. 1 OFTEN 2. 2 SOMETIMES 3. 3 RARELY 4. 4 NEVER 5. 9 DK/REF (Missing values) Students Using Computers in the Classroom (1997 Faculty Survey)

ASCP Workshop 04June04 LA076 SDSU Strategies for Building Faculty Community Reliance on most enthusiastic and technically advanced instructors who are already using computing and modeling in classes The Faculty Fellows program: Stakeholders: –College Deans - Specific support –Faculty - Compensation, and acknowledgement, of the value of the faculty members contribution Benefits –College –Department (Faculty Fellows as discipline-specific spokespersons for EC/CSE) –Faculty (as individuals) –Ed Center on Computational Science and Engineering –Building a special infrastructure for curriculum transformation: human, institutional, technical – is a requirement for successful introduction of advanced techniques (since they are more demanding on faculty time and efforts) The problems and strategies are not that much different from STEP!!

ASCP Workshop 04June04 LA076 SDSU Faculty Fellows during Faculty Fellows representing four departments from four colleges: Geological Sciences, Geography, Computer Engineering, Business Information Systems Bi-weekly meetings at the Ed Center Faculty Fellows as “ambassadors” of computational science Partnership with LEAD for evaluation during Follow-on Activities (Susan Millar, LEAD)LEAD –CATS (Classroom Assessment Techniques)CATS –FLAG (Field-tested Learning Assessment Guide)FLAG –SALG (Student Assessment of Learning Gains)SALG

ASCP Workshop 04June04 LA076 SDSU More Strategies Trying the new approaches in our own teaching: –Teaching the Supercomputer class with group-based problem-solving environments –Real-time distance teaching with Web-based collaborative software (featured as Microsoft Case Study in Higher Ed.) –Development of Sociology Workbench, a free on-line survey data analysis system that can be used for evaluation of student outcomes and other surveys

ASCP Workshop 04June04 LA076 SDSU Channels for Influencing Pre-Service Teacher Preparation Use of advanced computing modules in general ed courses (e.g. Geol 303 “Natural Hazards”) Cooperation with College of Education faculty and students, esp. in Education Technology: focus on experimentation with new technologies in classroom setting Computational Science Olympics: supporting the “bottom- up” development of computational science curricula Providing on-line assessment technologies: –Sociology Workbench:

ASCP Workshop 04June04 LA076 SDSU SWB Convenient Tool to Learn from Student Survey Data Online tool for “standard public data sets” or your own data set Small Sample, therefore only useful as feedback for the instructor Can be used with “forms” interface directly into SWB format, as in June ‘99 CSU Faculty Workshop

ASCP Workshop 04June04 LA076 SDSU SWB as Analysis Tool View Student Comments (text)

ASCP Workshop 04June04 LA076 SDSU SWB as Analysis Tool Isolate on Specific Survey Response

ASCP Workshop 04June04 LA076 SDSU SWB as Analysis Tool Explain the Response on Learning with “doing more”

ASCP Workshop 04June04 LA076 SDSU SWB as Analysis Tool Explain learning with “active participation”

ASCP Workshop 04June04 LA076 SDSU Lessons Learned Institutional support required for program to be sustainable Individual reform-ready faculty is focus for support Infrastructure: –Build a Synergistic Environment (across disciplines) for Faculty –Continuous monitoring through interviews, surveys, discussions

ASCP Workshop 04June04 LA076 SDSU What’s Next The approaches we described proved useful for two target audiences. We believe that the strategies and lessons learned can be extrapolated in a targeted effort to incorporate computational science technologies in pre-service teacher preparation This may be a funding opportunity? We will be happy to contribute our knowledge and share experiences…

ASCP Workshop 04June04 LA076 SDSU References ASCP Workshop User-Friendly Handbook for Project Evaluation: Science, Mathematics, Engineering and Technology Education, NSF Sid Karin: The Importance of Science Literacy in a Computing World (see enVision Sci ence Magazine, V.15 No. 2Importance of Science Literacy K. Stewart & I. Zaslavsky, “Ten Grand Challenges”, Supercomputing ’98 Orlando FL SC98 Conference PaperSC98 Conference Paper J. Foertsch & B. Alexander, “Integrating HPC into the Undergraduate Curriculum”, report by LEAD Center June

ASCP Workshop 04June04 LA076 SDSU Evaluation & Assessment U. Wisconsin LEAD for 1998/99 Ed Center evaluation Follow-on Activities (Susan Millar, LEAD) CATS (Classroom Assessment Techniques) FLAG (Field-tested Learning Assessment Guide) SALG (Student Assessment Learning Guide) CATS FLAG SALG

ASCP Workshop 04June04 LA076 SDSU Evaluation and Assessment of Classroom Practice Where to start? User-Friendly Handbook for Project Evaluation: Science, Mathematics, Engineering and Technology Education, NSF Student Surveys - Need a compatible tool for instructor to examine results with Sociology WorkBench (SWB) developed by team of undergraduate computer science majors employed by the EC/CSE

ASCP Workshop 04June04 LA076 SDSU Information Overload How can teachers help out? How can technology providers help out? Ans: personalize the linkage.

ASCP Workshop 04June04 LA076 SDSU Information Overlab Lab Using the WWW extensively in the classroom, implies obligation to discuss Information Overload Timeline of Technology

ASCP Workshop 04June04 LA076 SDSU Computer Network Security Campus-wide “buy-in” recognizing the use of high-speed networks the responsibility for “secure network use” SDSU Academic Senate adopts interim policy UCSD Acceptable Use Policy useful start

ASCP Workshop 04June04 LA076 SDSU A Little History - May 10, 1869 What was the major event? Thank you to the US Library of Congress for the on-line exhibit preserving our “Memory”

ASCP Workshop 04June04 LA076 SDSU

Promentory Point, UTAH

ASCP Workshop 04June04 LA076 SDSU How does this relate to us today?

ASCP Workshop 04June04 LA076 SDSU Technologies to Motivate Learning Mr. Sid uses Wavelets for compression of Railroad Maps Sense of History must be tied to personal, student- based, “stories” University Faculty need encouragement this will “count” for professional recognition by their peers and administrators (Deans…)

ASCP Workshop 04June04 LA076 SDSU ONLINE Resources for Undergrads The Education, Outreach and Training (EOT) of the PACIs feature several projects Ed Center on CSE Shodor Undergraduate Computational Physics Minority Institutions, a Directory Student Biology Workbench (Bioinformatics) Underrepresentation in Computer Science (CDC)

ASCP Workshop 04June04 LA076 SDSU Learning Technologies from the PACIs Data Intensive Computing Environment Digital Video/eTEACH Interaction Environments: Bays to Brains Neuroscience: Telescience/Tomography NISE College Level One Institute on Learning Technology Remote Collaborative Learning

ASCP Workshop 04June04 LA076 SDSU This is Only the Beginning... YOU ARE HERE TIME

ASCP Workshop 04June04 LA076 SDSU What’s Next Your comments and thoughts? SIAM Computational Science & Engineering Conference – Sept – Washington D.C. SC2000 – November 2000 – Dallas TX

ASCP Workshop 04June04 LA076 SDSU

NPACI Partnership Report enVision quarterly science magazine, especially June2000 “Why the Future May Very Well Need Us” “Online” biweekly electronic publication, NPACI Sources of Information

ASCP Workshop 04June04 LA076 SDSU References - EPSCoR - July 2000 This Presentation pthttp:// pt Ed Center on Computational Science & Engineering Timeline of Technology Development mlhttp:// ml Education, Outreach and Training Information Overload