Presentation is loading. Please wait.

Presentation is loading. Please wait.

Almost every academician has heard the term scholarship of teaching. However, exchanges among colleagues from many institutions reveal a heterogeneity.

Similar presentations

Presentation on theme: "Almost every academician has heard the term scholarship of teaching. However, exchanges among colleagues from many institutions reveal a heterogeneity."— Presentation transcript:

1 Almost every academician has heard the term scholarship of teaching. However, exchanges among colleagues from many institutions reveal a heterogeneity of meanings, attitudes, and experience levels with this topic. A comprehensive and coherent framework seems needed to accommodate the diverse strands of thought and endeavor. Similarly, a treatment of SOTL-related topics that accommodates a broad continuum of familiarity and experience seems called for. The tutorial attempts to meet those needs, by providing a primer … a useful grounding in the whole domain of SOTL- related topics. Materials for the tutorial originated in workshops and conferences sponsored by the American Association of Higher Education (AAHE), the Carnegie Foundation for the Advancement of Teaching, the Professional and Organizational Development Network in Higher Education (POD), the Lilly Conferences on College Teaching and various individual colleges and universities. Your hosts: Samuel B. Thompson, Craig E. Nelson, and Rita C. Naremore A group making the journey together along the continuum from Alpha to Omega will find interesting topics for discussion and rich resources for further study. The tutorial is well enough indexed to also serve encyclopedically for those who seek information about specific SOTL-related topics. The tutorial speaks to so many of the interests and concerns of faculty members and administrators who are grappling with issues of teaching, scholarly teaching, and the scholarship of teaching and learning. You and your colleagues have done a tremendous job in organizing and presenting these materials. Barbara Cambridge Welcome to a Tutorial on the Scholarship of Teaching and Learning (SOTL)

2 Using the Tutorial Click at any point on the screen to advance to the next frame. Move your mouse to show faint arrows at the lower left corner of the frame. Click on either of these arrows to show a menu. To move backward one frame, click on ( Previous ). Jumps to frames elsewhere in the tutorial are dark blue and underlined. To return to the frame from which you jumped, use ( Go, then Previously Viewed ). This two-step procedure will probably be frequently used. Click on ( Go then Slide Navigator ) to see a menu of frames and to move to any point in the tutorial. To end the tutorial, use ( End Show ) Other types of embedded resources linked to the main path are indicated by icons: Website Video Audio Respond to a task (click on icon) This is a multimedia tutorial including photos, graphics, hyperlinks to sources, audio and video clips, and interactive tasks as well as text. The tutorial is organized in frames (not pages) consisting of short blocks of information about the topic in the frame title. The continuum is traversed by proceeding serially along a main path through the frames, stopping to examine embedded resources whenever they seem of interest. There are many frames, and it is not intended that the viewer intent on mastery of content will travel through all of them in one, or even two sittings. However, we have tried to make the journey entertaining as well as informative. Travel through the frames goes quickly, information is conveyed in various ways, and many good stopping places are evident. Its really impressive to see how much you have compressed and organized into a manageable package. I confess I was a bit daunted by the number of frames at first, but the tutorial moves along very smoothly. A very efficient way to learn this stuff. Pat Hutchings

3 Acknowledgements Moya Andrews Thomas Angelo William Becker Simon Brassell Barbara Cambridge William Cerbin Donetta Cothran Patricia Cranton K. Patrica Cross Alix Darden Robert Diamond Donna Duffy David Pace Edward Redish Eugene Rice Laurie Richlin Leah Savion Anita Salem Donald Schon Nancy Shanklin Lee Shulman John Sullivan George Walker Barbara Walvoord IndividualsInstitutions Abilene Christian University The Citadel Elon College Indiana University Middlesex Community College Notre Dame University Rockhurst University Information and materials from more than 30 individuals across the United States and Canada as well as SOTL program information from seven colleges and universities are used in this tutorial. All contributions are gratefully acknowledged. Charles Glassick Samuel Guskin Mary Huber Pat Hutchings Robert Hutchins Dennis Jacobs T. Mills Kelly Carolin Kreber Marcia Landen Suzanne Mabrouk Gene Maeroff Valerie OLoughlin This project is sponsored in part by an American Association of Higher Education (AAHE) Grant © 2001 Samuel B. Thompson

4 1. How would you describe your experience with scholarship of teaching and learning? Choose one: Extensive Moderate Hardly-Any None 2. OUTCOMES: List 3 topics or issues you hope to learn more about from this tutorial STOP NOW! BEFORE YOU GO FURTHER, WE NEED YOUR RESPONSES TO THIS FRAME. This task is DESIGNED to capture experience level of users and their expectations/goals in freeform before influence by tutorial content. If you will be so kind as to activate the button in the slide and type your responses, well put the information you provide to good use! Task A

5 Boyer coined the term scholarship of teaching more than ten years ago. The American Association of Higher Education (AAHE) and the Carnegie Foundation for the Advancement of Teaching have conducted initiatives to advance this scholarship for several years. Now, a continuum of experience levels is evident among academicians. At the Alpha end of the continuum, people have heard the term but are unsure what it means. These need perhaps a glossary and basic grounding in the key developments over the last decade. In the middle are people who may have launched campus initiatives or may be wondering whether particular issues and projects could result in scholarly products. At the Omega end, those at the cutting edge of SOTL endeavor to construct and validate developmental processes as sustainable bridges to carry faculty from initial interest to stages of productivity that achieve goals and count in reward systems. This tutorial addresses the whole continuum of SOTL experience from Alpha to Omega. The three sections of the continuum correspond to the three units of this tutorial and each phrase names a module … so two modules per unit, six total. The first unit, features essential definitions, concepts, rationale and other background. The second unit features administrative how to and examples of campus programs as well as faculty projects. The third unit provides some research essentials to permit viewers to launch their own projects. The metaperspective afforded in considering the whole continuum permits viewers to structure their own projects and campus initiatives. ΑlphaOmega 1A What & Why of SOTL 1B Origins of SOTL 2A Initiating SOTL Programs 2B Faculty SOTL Projects 3A Bridges to Productivity 3B Questions, Designs & Resources Continuum of Experience with SOTL A Metaperspective

6 Alpha to Omega Continuum:Contents AlphaOmega Unit 1A The What of SOTL Why SOTL? Task B: Reasons to Engage Unit 1B Overview Reform Concepts Implementing Entities Recent Articulations Task C: Critique of Origins and Evolution Unit 2A A Campus Example in Detail Task D: Refining Your Campus ProgramTask D: Refining Your Campus Program Examples on Other Campuses Task E: Applicable Features Unit 2B Donetta Cothran – Kinesiology Valerie OLoughlin – Medical Sciences David Pace – History William Becker - Economics Rita Naremore – Speech and Hearing SciencesRita Naremore – Speech and Hearing Sciences Dennis Jacobs – Chemistry Leah Savion – Philosophy Other Examples – Carnegie Scholars Task F: Minute Paper Unit 3A How Could I Do SOTL (Genres) Task G: Reflecting on Genres Approaches to Scholarship Via Classroom Research ProjectsApproaches to Scholarship Via Classroom Research Projects Where to Publish and Present Sources of External Funding Task H: Classroom Research Unit 3B Framing Questions Task J: Examining Valeries Questions Task K: Framing Your Question Designs for Studies Choosing Measures Guiding Questions in Choosing MethodologyGuiding Questions in Choosing Methodology Task L: Designing Your Project Summary of Standards

7 Unit 1A The What and Why of SOTL We seek understanding of what SOTL is and its relationship to teaching as well as reasons for becoming engaged in SOTL. The What: A Conceptualization of Teaching-Related ActivitiesThe What: Why SOTL? Task B: Your reasons to become engaged in SOTLTask B

8 Many conceptualizations of SOTL have been offered over the years. This one is ours. Others will be exhibited in the next module. Every teaching-related act can be represented as a point somewhere within the three elliptical regions. Elliptical shapes are chosen solely for convenience of representation. All deeds that teachers perform in the name of carrying out their teaching assignments can be imagined as individual points within the large ellipse labeled teaching. The centillions of such points include acts of materials development, grading, individual tutoring … every teaching activity in or out of the classroom. Within this big red ellipse of teaching acts, we can identify a subset that meet special criteria for scholarly teaching. The smallest ellipse (yellow) represents scholarship of teaching and learning. Scholarship of teaching and learning (SOTL) is rooted in teaching but extends beyond it (out of the red ellipse). SOTL involves creation and dissemination of original work that makes a useful contribution to knowledge and practice of other teachers. Though sharply defined in the picture, the boundaries of ellipses can seem a bit fuzzy when classifying certain teaching-related acts. This occasional fuzziness does not lessen the usefulness of this seemingly simplistic conceptualization. More about Lee Shulman More about Pat Hutchings A Conceptualization of Teaching Related Activities

9 Faculty members should be encouraged to move acts of teaching that are characterized by points in the red toward the orange … in other words toward scholarly teaching. To the extent that teachers adopt practices of scholarly teaching, their teaching will probably improve. Similarly, their acts of scholarship will probably improve their teaching, develop them professionally, and add to our body of useful knowledge about practice. However, we must stop short of accepting either the proposition that scholarly teaching implies excellent teaching or the converse. Similarly scholarship of teaching does not imply excellent teaching. Excellence is often confounded with scholarly teaching and scholarship of teaching. Scholarly teaching practices do not necessarily confer excellence. All seasoned faculty know of colleagues who are excellent teachers … excellent in terms of achieving superb outcomes in students … without their meeting any of the criteria of scholarly teaching Avoidable indignation and opposition to SOTL initiatives are justifiably engendered in faculty by the notion that scholarly teaching and scholarship of teaching are necessarily what one must do to be considered excellent in teaching. No ascending hierarchy of excellence can necessarily be associated with movement from left to right or bottom to top in the Venn diagram. To represent excellence, we must invoke a third dimension perpendicular to the plane of the diagram. In this three-dimensional conception, each teaching-related deed represented by a point in the diagram has associated with it an arrow (a quality vector) representing excellence (or lack of it). The direction of the arrow indicates whether the deed is above or below average in quality and the length of the arrow indicates how far above or below average. Relationship to Excellence

10 Into what realm beyond teaching does scholarship of teaching extend? The rightmost ellipse is the set of points representing deeds that scholars perform in the name of research and creative activity. Though research-based activity to advance knowledge is the prevailing notion of scholarship, George Walker tells us that scholarship is not precisely synonymous with research. Scholarship is reflective and creative. It can result from reflecting on the processes and outcomes of research or personal teaching experience and creating something new.George Walker scholarship is not precisely synonymous with research. Boyers notion that scholarship of teaching and learning spans the tired teaching vs. research debate is naturally depicted in the position of the yellow ellipse (with italicized text) between the two big red ellipses. We can distinguish between SOTL as an academic activity and as a campus initiative. As an academic activity, what scholarship of teaching and learning means in your institution defines the interior and boundary of the yellow ellipse. As a campus initiative, what SOTL means may encompass the entire 3 leftmost ellipses. In other words, a SOTL initiative may include all teaching related activities of faculty. Thusly conceived, a SOTL initiative can provide a resource-rich and opportunity-laden home to faculty members who aspire SIMPLY to teach excellently as well as to those who perceive themselves primarily as scholars (i.e.researchers). In fact, there is evidence that faculty who have not been attracted to previous teaching improvement initiatives can be attracted to a SOTL initiative because of its research orientation. This phenomenon may be visualized in the diagram as faculty converging to SOTL not only from the big red ellipse on the left but also from the one on the right. SOTL as Academic Activity vs. Campus Initiative

11 Unite Where Teaching and Pursuit of New Understanding are Shared Interests Build Multidisciplinary Community Capture the Wisdom of Practice Foster Mentoring of New Generations of Teachers Employ Reserve Intellectual Capacity Recreate the Concept of Scholarly Excellence Elaboration of the unifying potential of SOTL (Voice of Samuel Thompson) Elaboration of community-building via SOTL (Voice of Samuel Thompson) Documentation of the deep and useful insights gained over time so that others can use them is encouraged by SOTL. The documentation includes the teaching moves and materials that every seasoned faculty member has found to work especially well. SOTL is a medium for the generativity that develops naturally in mid-life … seasoned teachers helping newer ones to learn to teach more effectively. SOTL may increase the individual and collective scholarly productivity of the faculty, partly by providing an avenue to products from the immense investments that already go into teaching and partly by providing an area where faculty productivity can thrive when scholarly efforts in other areas are flagging or no longer interesting. SOTL brings the spirit and standards of scholarly excellence to teaching.. standards that have traditionally been applied to other fields of scholarly endeavor but not to teaching. SOTL can improve teaching and learning … and more! SOTL has power to: Why SOTL?

12 Why SOTL? (Cont.) Knefelkamp L. L. (1990) Seasons of Academic Life: Honoring Our Collective Autobiography Liberal Education Power of Ideas Faculty Role Students Public Self Multiple and Competing Commitments Need to Step Out Marginality Courage SOTL Honors and Enriches the Seasons of Academic Life Elaboration of the role of SOTL in the seasons of academic life (Voice of Craig Nelson) Elaboration of the role of SOTL in the seasons of academic life (Voice of Craig Nelson) Knefelkamps characterization of Seasons of Academic Life seems worthwhile reading for any faculty member. This particular article may serve as a basis for a meaningful exercise in deepening understanding of Why SOTL? The recommended exercise is to read Knefelkamps description of each season and then pause to reflect on the possible roles that scholarship of teaching and learning may take during it and the implications for the faculty member.

13 What reasons to become engaged in SOTL … the ones discussed in the tutorial or others … are most important on your campus? What reasons might be irrelevant to faculty on your campus? We pause here for processing of the foregoing and for reflection. Please activate the button and your reasons. Well compile categories of reasons and disseminate the results. Task B: Reasons to Engage in SOTL

14 Unit 1B Origins and Evolution of SOTL Overview Reform Concepts Implementing Entities Recent Articulations Task C: Critique of Origins and Evolution As we progress to the second and subsequent modules, well continue to broaden and deepen our concept of what SOTL is and why become engaged in SOTL. We invite your peer review and critique of this module. At the end of the module, we will ask you (Task C) to tell us how you would modify the framework and which elements you would add or omit in each the first four categories above. Well consider your inputs and may incorporate them in future versions of this tutorial in order to build a better framework for everyone.

15 Not a new idea (Hutchins 1923) Scholarship of teaching coined (Boyer 1990) Reform Concepts Classroom Assessment/ Research New Epistemology New American Scholar Conception of Teaching Scholarship Assessed Implementing Entities AAHE Carnegie Pew Lilly Recent Articulations Scholarship of Teaching Scholarly Teaching Model of SOTL Relation of ST to SOTL Origin and Evolution of SOTL (In Progress) The work in progress in this tree is our tentative attempt to build a succinct framework for essential background in SOTL. Of course, the number of individuals, forums, articles, and books that have significantly contributed to SOTL is too enormous to include comprehensively. And who is to say which of these is essential? So this whole exercise is somewhat arbitrary and no doubt reflects the bias of our own limited knowledge and experience.

16 What is certain is that most Ph.D.s become teachers and not productive scholars as well. [A Ph.D. candidate who plans to be a teacher] must know his field and its relation to the whole body of knowledge. It means too that he must be in touch with the most recent and most successful movements in undergraduate education, of which he now learns officially little or nothing. How should he learn about them? Not in my opinion by doing practice teaching upon the helpless undergraduate. Rather he should learn about them through seeing experiments carried on in undergraduate work by the members of the department in which he is studying for the degree…. Robert Maynard Hutchins The quote is taken from Hutchins inaugural address as the fifth president of the University of Chicago in 1928 and is presented as evidence that the concept of SOTL existed long before Boyer popularized the term. The last sentence suggests that faculty in all departments should carry on experiments in undergraduate teaching and learning … a fundamental activity of SOTL. The sentence further suggests that graduate students preparing for academic careers should learn from these experiments. Note that Hutchins also presaged the plight of many undergraduates, and the need for programs to prepare graduate students to teach … programs like the Preparing Future Faculty (PFF) initiative of the 1990s.Preparing Future Faculty In the first sentence, Hutchins also expressed a truth still not internalized by many faculty who prepare graduate students in doctoral institutions today … research productivity that advances the discipline ends with the dissertation for most Ph.Ds. Today, the vast majority (95% is a figure often quoted) of new Ph.Ds who take positions in academe do so in teaching institutions rather than research universities. In other words, though they may acquire their Ph.Ds in research universities as researchers, teaching is what they will primarily do in academic careers. Not a New Idea

17 Scholarship of Teaching Coined Ernest Boyer popularized scholarship of teaching in a famous monograph Scholarship Reconsidered: Priorities of the Professoriate Princeton, NJ: The Carnegie Foundation for the Advancement of Teaching. Boyer argued that the concept of scholarship must be broadened to include not only basic research but other kinds of intellectual work in which faculty engage. He suggested four types of scholarship: discovery, integration, application, and teaching. All of the elements of Reform Concepts build on Boyers theme of broadening the traditional view of scholarship.Reform Concepts Boyer disseminated the notion of a scholarship of teaching but did not define clearly what this scholarship would be. He thereby touched off a decade of academic thought and controversy over this topic. As a result, our conceptualization of this scholarship is a bit clearer today (but only a bit!)

18 Reform Concepts: Classroom Assessment/ Classroom Research Reform Concepts The several reform concepts presented in this section of the tutorial may be viewed as multiple origins or histories of SOTL. They are somewhat overlapping and sometimes difficult to isolate. Classroom Assessment Techniques (CATs) K. Patricia Cross and Thomas Angelo pioneered this do-it-yourself approach to assessing conditions of learning in classrooms. Their 1993 book, Classroom Assessment Techniques: A Handbook for College Teachers (2nd Ed) Jossey- Bass is a famous work of scholarship of teaching and learning, reported to have sold more than 35,000 copies. Thousands of college and university teachers use CATs in their classrooms today, especially versions of the one-minute paper, and there is evidence of improved learning as a result. Angelo and Cross contributed to the current focus on learning but also hoped their book would influence faculty to go beyond application of CATs in the classroom … that it would in fact spark systematic classroom research which would in turn become a popular form of SOTL nationally … a form more useful to faculty than traditional educational research. The adoption of CATs became widespread, the adoption of classroom research much less so.evidence of improved learning as a result Classroom Research In 1996, Cross with Mimi Steadman published a sequel book, Classroom Research: Implementing the Scholarship of Teaching, Jossey-Bass placing CATs in a broader research context. Again the hope was to institutionalize classroom research. However, as of yet, classroom research is still not nearly as widely employed as are CATs. More on CATS More on Classroom Research

19 Reform Concepts: New Epistemology On the high ground, manageable problems lend themselves to solution through the use of research-based theory and technique. In the swampy lowlands, problems are messy and confusing and incapable of technical solution....the problems of the high ground tend to be relatively unimportant to individuals or to the society at large, however great their technical interest may be, while in the swamp lie the greatest problems of human concern....Shall [the practitioner] remain on the high ground where he can solve relatively unimportant problems according to his standards of rigor, or shall he descend to the swamp of important problems where he cannot be rigorous in any way he knows how to describe? Donald Schon Donald Schon posed a dilemma of rigor or relevance for research epistemology. In this eloquent quote, he calls for new research epistemology for the scholarship of teaching and learning based on reflection by teachers on actions taken in their practice. Schon, D. The New Scholarship Requires a New Epistemology, Change, Nov./Dec ( 6) p. 28

20 Reform Concepts: The New American Scholar Adapting the learning cycle of David Kolb to SOTL, Gene Rice sees traditional scholarship … particularly the so-called social science model … as characterized by reflective observation and abstract analytic knowing. It therefore resides in the lower right quadrant of his diagram. He asserts that SOTL needs to be rooted more in active practice and concrete connected knowing. Said another way, scholarship of teaching and learning should take forms that properly reside in the upper left quadrant

21 Reform Concepts: Conception of Teaching & Scholarship Assessed Conception of Teaching: The argument here … and Lee Shulman is a major proponent ….is that teaching is not just technique (though technique has gotten the lions share of attention in teaching-improvement efforts) but an enactment, rather, of our understanding of our disciplinary, interdisciplinary or professional field and what it means to know it deeply. As Shulman wrote recently "a scholarship of teaching will entail a public account of some or all of the full act of teachingvision, design, enactment, outcomes, and analysisin a manner susceptible to critical review by the teachers professional peers, and amenable to productive employment in future work by members of that same community." Scholarship Assessed: Glassick, Huber, and Maeroff (1997) in Scholarship Assessed: Evaluation of the Professoriate. Jossey-Bass review the history of scholarship in American academe and describe general standards for all forms of scholarship. These standards pull together many of the concepts noted in this section. More detail on the standardsMore detail on the standards Summary of Reform Concepts. All reform concepts contain an implicit call to broaden and generalize traditional notions of scholarship in order to make progress against relevant problems which vex undergraduate teaching and learning at levels of individual teacher and course. George Walker advises us to avoid unnecessary limitations in our notions of scholarship. The goal is to produce a body of scholarship that may deviate from conventional forms, and that may not be applicable over disciplines, categories of student, or institutional type but which nevertheless is more used by teachers and contributes more to the learning of students than does past educational research.George Walker avoid unnecessary limitations in our notions of scholarship.

22 Implementing Entities: AAHE AAHE (American Association of Higher Education) AAHE (American Association of Higher Education) The AAHE has sponsored many initiatives over the years to advance teaching and its scholarship. Barbara Cambridge directs the Teaching Initiatives. Major initiatives include assessment, service-learning, peer- review of teaching and teaching-learning technologies. We highlight faculty roles and rewards below because its historic origin is associated with Boyers introduction of scholarship of teaching. AAHE FFRR (Faculty Forum Roles and Rewards) It was recognized at the time of Boyers 1990 monograph, that overhaul of the faculty reward system needed to go hand-in-hand with advancement of scholarship of teaching, if the latter was to be more than a transient topic. The FFRR has met annually since 1990 to address faculty roles and rewards. Robert Diamonds depiction of the factors bearing on faculty work reflects the agenda of AAHE FFRR. Initiatives have been undertaken in the 1990s to favorably influence each of the five factors in the figure toward greater incorporation of SOTL in faculty work. Factors Influencing What Faculty Do Implementing Entities We focus on broad national initiatives. There are far more entities (including many academic institutions) deserving acknowledgement for implementation than those included here. Diamond, R.& Adams, B. The Disciplines Speak AAHE 1995 p.7

23 Implementing Entities: Carnegie THE CARNEGIE FOUNDATION FOR THE ADVANCEMENT OF TEACHING The Carnegie Foundation has a number of exemplary programs and resources that encourage scholarly teaching and the scholarship of teaching and learning. Among the programs that may be explored are: Cultures of Teaching and Learning in Higher Education, Higher Education and the Development of Moral and Civic Responsibility (co-led by Tom Ehrlich, former President of Indiana University), Preparation for the Professions, Teacher Education, and the long running U.S. Professors of the Year. Four key programs to click on:programs that may be explored Rethinking the Doctorate. George E. Walker, Vice President for Research and Dean of the University Graduate School at Indiana University, leads this new, five-year project.George E. Walker Knowledge Media Laboratory. A gallery of course portfolios illustrating various approaches to the scholarship of teaching. Relation between course portfolios and SOTLRelation between course portfolios and SOTL Resources. Click on e-Library to read key SOTL references on-line and to access an annotated bibliography of SOTL in higher education. CASTLCASTL Carnegie Academy for the Scholarship of Teaching and Learning

24 Pew National Fellowship Program for Carnegie Scholars For faculty from higher education - brings together outstanding teacher-scholars from a variety of academic fields and institutions to pursue investigations of issues in teaching and learning. Teaching Academy Campus Program Coordinated for the Carnegie Foundation by AAHE. Institutions in higher education that are prepared to make a public commitment to the scholarship of teaching and learning are invited to join the Campus Conversations program. As progress is made, campuses are invited to apply for small grants for the Going Public phase. CASTL now is also encouraging the formation of National Networks of Teaching Academy Campuses. The AAHE CASTL Campus Program Web Center includes a number of resources for thinking about SOTL, including reports from campus conversations, a list of going public grants and selected resources.AAHE CASTL Campus Program Web Center Work with the Scholarly and Professional Societies Carnegie is encouraging societies to increase their emphasis on teaching and on the scholarship of teaching and learning. This is a continuation of work under AAHE FFRR culminating in two publications which contain perspectives of disciplinary societies on Boyers four scholarships: The Disciplines Speak AAHE, 1994 and The Disciplines Speak II AAHE, 2001.The Disciplines Speak II AAHE, 2001 Carnegie Academy for the Scholarship of Teaching and Learning (CASTL)(CASTL) Implementing Entities: Carnegie (cont.) CASTL

25 Implementing Entities: Pew Peer Review of Teaching Course Portfolios Pew Charitable Trusts: CASTL (with Carnegie Foundation) Peer Review of Teaching Peer Review of Teaching is an ongoing area of work intertwined with the scholarship of teaching and learning in several ways. Peer Review Peer Review has been an emphasis of AAHE since 1994 and continues to be supported by the Pew Foundation and other grant sources. The course portfolio has evolved as the principal means by which peer review of teaching is effected. Two notable current programs to advance course portfolios, both supported by Pew, are centered at the University of Nebraska Lincoln and Samford UniversityUniversity of Nebraska LincolnSamford University Individual course portfolio projects and SOTL projects are so closely related as to often seem inseparable. Example of this inseparability.Example of this inseparability. Perhaps the best way to ensure that quality teaching is recognized, valued, and rewarded is to improve the means of identifying and documenting teaching effectiveness. Course portfolios afford a comprehensive yet efficient means of documenting the intellectual work of teaching a particular course. Through such a portfolio, a faculty member documents course design and execution, including results in student learning. In this way, teaching can be understood and presented as a form of scholarship, utilizing the accountability through peer review that already exists in higher education. A course portfolio can be used as an instrument for exhibiting teaching effectiveness, a framework for cultivating scholarship, and a vessel for conveying one's work to appropriate publics, including promotion and tenure committees.

26 Implementing Entities: Lilly Conferences on College Teaching Lilly Conferences Lilly Conferences are retreats that combine workshops, discussion sessions, and major addresses, with lots of opportunities for informal discussion about excellence in college and university teaching and learning. Internationally-known scholars join new and experienced faculty members and administrators from all over the world to discuss topics such as gender differences in learning, incorporating technology into teaching, encouraging critical thinking, using teaching and student portfolios, implementing group learning, and evaluating teaching. The welcoming "Lilly Spirit" and the high level of scholarly attention to teaching and learning enable everyone to contribute to the forum. Lilly Conferences have encouraged scholarly work in teaching and provided a venue for faculty members to discuss and disseminate such work for more than 20 years.

27 Recent Articulations: Scholarship of Teaching Scholarly Teaching We have left the topic of Implementing Entities, and moved to Recent Articulations (the rightmost branch of the origin and evolution tree. ) A decade after Boyers Scholarship Reconsidered, the meaning and implications of scholarship of teaching and learning are topics of evolving thought and discussion. Many more entries than the four shown could be included in the right branch and these would be no less worthy than the sources shown. However, our purpose is to provide a sample rather than a comprehensive list. Our apologies to authors for failing to list their contributions.origin and evolution tree Scholarship of Teaching and Learning This recent articulation (Shulman,Change 31 (4) 1999) is widely accepted. Note that it is not quite a definition; it offers necessary conditions for scholarship but not sufficient conditions. The articulation does suggest a broadening of the concept of scholarship as advocated in the Reform Concepts Applications of the broadened concept of scholarship.Reform ConceptsApplications of the broadened concept of scholarship Scholarly Teaching This articulation (Hutchings & Shulman Change 31 (5) 1999) is also widely accepted. One of its principal features is to include knowledge of recent and relevant developments in pedagogy as well as the traditional notion of recent and relevant developments in the field as a criterion for scholarly teaching.

28 Recent Articulations: Exploring Scholarship of Teaching The Model of Kreber & Cranton Carolin Kreber and Patricia Cranton see development of scholarship of teaching as a process of reflection on experience and research in teaching. The second, third and fourth concentric circles represent content, process, and premise reflection respectively. The three spokes partition the circles into regions representing the three domains of knowledge (instructional, pedagogical, and curricular) shown on the periphery of the figure. Within each knowledge domain, learning is characterized by various forms of instrumental, communicative, and emancipatory learning processes, which are derived from Habermass three forms of knowledge. Development of SOTL in this conceptualization is thus comprised of nine distinct types of learning.

29 Recent Articulations: Conceptualizing Scholarly Teaching and Scholarship of Teaching Laurie Richlins conceptualization reflects her extensive experience as co-director of the Lilly Conferences on College Teaching and in helping faculty with teaching-related investigations. Her conceptualization shown in the slide is closely tied to the ideas of classroom research. She partitions a developmental process for SOTL according to the aspects that can be considered scholarly teaching and scholarship of teaching. Her steps in the part of the process called scholarly teaching are similar to those in our classroom research schematic.classroom research schematic Richlin, L in Kreber, C. (Ed.) New Directions in Teaching and Learning (In Press)

30 Task C Think about your own knowledge of SOTL. What additions would you make to the framework Origins and Evolution of SOTL in each category?Origins and Evolution of SOTL Reform Concepts Implementing Entities Recent Articulations We pause to process, reflect on, and improve the Origins and Evolution of SOTL. WE NEED YOUR HELP WITH THIS! Please suggest alterations to the structural framework of the tree as well as to the specific elements in the branches. Just click on the button on the frame to your suggestions. Well be most grateful for your contributions, and they will result in a better product for everyone.

31 Unit 2A Initiating SOTL Programs A Campus Example in Detail ( Indiana University Bloomington)A Campus Example in Detail Task D: Refining Your Campus Program Examples on Other Campuses Task E: Applicable Features We are leaving the left region (Unit 1) of the Alpha to Omega Continuum and transitioning to the middle region (Unit 2).Alpha to Omega Continuum

32 Identifying Key Resources Getting People Involved Changing the Institutional Culture Assessing SOTL Program Impact Task D Refining Your Campus Program An alternative path through this unit is to view Assessing SOTL Program Impact first. After studying A Campus Example in Detail, you will be asked to reflect on whether any aspects of this example are applicable to your own campus (Task D). A Campus Example in Detail: Using SOTL to Make Change Happen

33 Three Stages of Campus Program Goals Participation Scaffolding Scholarly Productivity Campus program development may be viewed in a framework of three stages. The first stage is to build awareness and participation. The second is to erect essential scaffolding to support faculty in this line of work. The third is to maximize scholarly productivity. It may be helpful to bear the above three stages in mind while studying this detailed example of a campus program. To assess the extent to which goals of the first two stages are achieved, look for evidence distributed through the frames of this unit. Evidence of scholarly productivity will be largely found in the following unitFaculty SOTL Projects.Faculty SOTL Projects

34 The Ingredients: –Massive administrative support –A dedicated director –A faculty advisory council –A core of faculty members who are willing to engage in this work and share it with others Moya Andrews Dean of the Faculties and Vice Chancellor for Academic Affairs What are the Key Resources? Massive administrative support Moya Andrews is committed to the campus SOTL initiative and an architect of it from its inception. Her administrative support is massive and visible to all. She provides a budget for the SOTL initiative. However, she also contributes vitally by staying abreast of SOTL-related developments and directing the campus initiative. She makes public statements, writes messages to faculty and invitations to department heads and other key individuals. She often introduces featured presenters at events. Message to faculty regarding SOTL

35 Massive Administrative Support (Cont.) George Walker Vice President for Research and Dean of the Graduate School Expanding our Vision of Indiana Universitys Research Mission: The Scholarship of Teaching and Learning Vice President Walkers talk at the first SOTL main event was subsequently delivered to representatives of research universities at invitation of AAHE and distributed nationally. Other SOTL resources by George Walker: A statement that SOTL is important.statement that SOTL is important A brief message to faculty regarding the rationale and significance of SOTL.brief message to faculty The campus SOTL initiative received a huge boost in administrative support when George Walker became a partner in it. From that time, the SOTL initiative was impelled by the two primary academic offices on the campus. This partnership facilitated the movement of faculty interest toward SOTL from research as well as teaching. George Walker describes some of the administrative support available for SOTL.administrative support available for SOTL George Walker is currently a senior scholar at the Carnegie Foundation leading a new study: Rethinking the Doctoratesenior scholar George Walkers contributions exemplify what a distinguished researcher and senior administrator can do to help advance teaching and its scholarship.

36 A Dedicated Director A campus SOTL initiative needs a person charged with responsibility to take actions on ideas, to follow up actions to ensure completion, to convene groups, to plan and stage events, to coordinate faculty and administrative viewpoints, to cheerlead, and to serve as a stable point of contact. If the director is a faculty member, release time is essential. The SOTL initiative needs to be a significant part of the directors portfolio. Samuel Thompson Director of the Campus SOTL Initiative to Samuel

37 Insure Representation Across Disciplines Tap Respected Faculty Let FACULTY Define Initiatives Support FACULTY Initiatives Success may depend on the initiative being of, by, and for the faculty. Advisory council members can steer the enterprise. They can be inspired by the prospects of what the initiative can do and so contribute their own time and effort without acknowledgement in tasks which they, themselves, often define. This is bottom up infusion of energy. Advisory council members should be chosen carefully. Ensure membership is representative. Primarily use senior, respected faculty: they know the ins and outs of the institution and their presence validates and lends prestige to the SOTL effort. Make sure the initial SOTL thrusts on the campus reflect consensus of faculty on the advisory council. Let faculty members feel empowered to influence key decisions. Advisory council members suggested doubling the amount of the SOTL grants proposed by administration, defined and addressed the human subjects issue for SOTL, and took the initiative to bring the research establishment into partnership. Administration supported the advisory council initiatives and decisions. The Faculty Advisory Council

38 A Core of Faculty Members Willing To Engage in This Work and Share It With Others Because this unit of the tutorial deals with launching campus SOTL initiatives, much emphasis is on administrative considerations. However, all the administrative brilliance in the world will not attract or sustain faculty in a SOTL initiative if they do not see relevance to academic issues important to them. If a campus already has scholars of teaching and learning, these may provide examples or kernels of issues which engage colleagues. But what if such scholars and issues are not readily identifiable? Engaging faculty should not be a struggle. The vast majority of faculty have a natural curiosity about teaching issues and a latent desire to do teaching-related inquiry. Making time and resources for reflection and inquiry available to faculty in light of their teaching loads and other responsibilities might be a challenge but as Shulman has suggested, students already provide faculty with more information than they normally use. Thus additional work, if any, in information-gathering should be minimal. A possible two-step procedure is to conduct a kind of needs assessment … formal or informal … to identify campus issues for possible investigation; e.g. academic incivility. The second step is to connect an identified need to language and issues that people care about … framing the issue to make it attractive in the particular setting. If engaging faculty in issues and framing them is of concern to you, you might keep this concern in mind when viewing other examples of campus programs, the issues investigated in Faculty SOTL Projects, and Framing the Question.other examples of campus programs Faculty SOTL Projects Framing the Question

39 Get Out the Crowd to Initial Events Showcase Campus Scholars Identify Campus-wide Needs and Interests Partner with Stakeholders Identify and encourage faculty who wish to pursue scholarly inquiries into teaching or have already done so. Make these visible as models and mentors in high profile events rather than bringing in outside speakers. Kick off the SOTL initiative with a Celebration of Teaching. Send personal invitations to winners of teaching awards, recipients of teaching-related grants, teachers of pedagogy courses for graduate students, and all others who have demonstrated interest in teaching in an identifiable way. Send personal invitations also to deans, chairs, and P&T committee members. About 200 faculty showed up for our Celebration, more than in anyones memory for a teaching-related event. Personal attention has continually proved important in building a SOTL community one faculty member at a time. Use information-gathering opportunities to learn the needs, goals and activities of people and units across the campus. How can SOTL help to fill their needs? Focus on issues for investigation that cut across disciplinary boundaries. Seek to make the SOTL initiative inclusive of all academic and non-academic units on the campus. Seek to involve undergraduate and graduate students, professional staff in academic affairs and student affairs, librarians, and faculty in all seasons of academic life. Seek as well to support and cooperate with all other teaching-related initiatives rather than to compete with them. Think of ways and themes for collaboration that advance the goals of stakeholders as well as those of the SOTL initiative. Getting People Involved

40 Getting People Involved: An Example As an example of partnering with stakeholders, consider librarians. They generally see themselves as academics but do not always feel that others see them in that light. When invited to join and help grow the SOTL initiative, they were very pleased for the opportunity to participate with faculty in an obviously academic enterprise. The librarians enriched SOTL with talent, initiatives, and resources. Librarians: volunteered as consultants to individual faculty investigators needing assistance in locating unfamiliar teaching and learning literature. gave presentations to show use of information technology to access SOTL-related resources. made free document delivery systematically available to scholars of teaching. acquired multiple copies of critical books and put them on reserve for SOTLites. served on working committees of the Faculty Advisory Council. The librarians, in turn, found participation rewarding both individually and collectively.

41 Getting People Involved (continued) This campus model as well as all six campus SOTL models described later in this tutorial draw direction and resources from AAHE and Carnegie national initiatives … and frequently publicize the national connections to the local community. Other connections to related national initiatives include Peer Review, Problem-Based Learning, and Preparing Future Faculty. Exhibit Administrative Support Its important not only to have administrative support but to exhibit it. One way is by regular participation of key administrators in SOTL events. Serving food exhibits institutional backing with funds as well as increasing participation, especially for events held at lunchtime or the end of the working day. Examples of SOTL eventsExamples of SOTL events Provide Scaffolding for Faculty If a SOTL initiative is to achieve its objectives, getting faculty interested or involved is not enough. Many need scaffolding … various forms of support … on the pilgrimage from initial interest to completion of scholarly work. Planning for this scaffolding and building it into the campus model is essential. Connect with National Initiatives

42 Change the faculty members view of what faculty members do Under the aegis of SOTL, employ administrative resources to acknowledge and reward a fuller range of faculty work. Expand the range of activities that get rewarded Many campuses have faculty who have done scholarship of teaching for years. However, nobody necessarily knows who they are if such scholarship does not count in reward systems. In this campus example, small grants ($1000) were offered at the start of the SOTL initiative for scholars of teaching to present work already completed or well underway. These unusual presentation grants for work that may have already been done made existing scholars and their projects visible campus- wide as models. Later conventional research grants to undertake new investigative projects were offered. Many campuses already have small grant programs. These can be tweaked to foster SOTL. A spin-off benefit can be greater productivity for a given institutional investment in grants; e.g. a grant that was previously to implement a teaching innovation will, as a SOTL project, also require data-gathering and analysis to assess the effect of the innovation as well as dissemination of results to peers. Expand the range of rewards Opportunities to present their work to large interdisciplinary audiences in high profile campus-wide events; to have their abstracts, biographical sketches, and photographs widely disseminated; and to experience the support and confidence of the central administration proved enough reward to some faculty members for them to offer themselves as SOTL presenters(without grants). For some faculty members, finding a community that valued their work to improve teaching and that helped them find opportunities for professional growth and recognition beyond the campus and beyond opportunities available in their departmental and disciplinary cultures also proved rewarding. Changing the Institutional Culture

43 IU FACULTY SUMMARY REPORT PRIOR TO ACADEMIC YEAR TEACHING ACTIVITIES A.Courses taught (weekly contact hours reported by course number in tabular form). B.Development or major revision of course(s) during the year. C.Dissertation, Research and Field Work Committees D.Teaching awards and honors, including those of your students. Indiana Universitys summary report form for individual teaching activities required of each faculty member annually was as shown above prior to Fall It is similar to that used in many institutions. There are also sections (not shown) for Research & Creative Activities and Service. Look at the categories of reportable teaching activities. Category B is generally taken to mean curriculum development or major revision of curriculum. Activities associated with improvement of pedagogy or with demonstrable evidence of improved student learning are not encouraged on such a report. Indeed, the lack of a place to report pedagogical activities on the form is a tacit statement about institutional values and rewards. Changing Institutional Culture: One Example

44 TEACHING ACTIVITIES A.Courses taught (weekly contact hours reported by course number in tabular form). B.Activities directed at improving instruction, learning, or course administration. (Please describe rationale for/description of innovations, methods/measures for assessing outcomes, and results.) Please note: Scholarly activity related to teaching and learning (e.g. investigation/research, dissemination/publication of results) should be reported under the section on Research/Creative Activities. IU FACULTY SUMMARY REPORT (REVISED) IN ACADEMIC YEAR AND THEREAFTER In Academic Year , the Faculty Summary Report form for the Bloomington campus was revised to include the new Category B shown in red. The former categories B, C, and D respectively become C, D, and E on the new form. This revision was prompted by the SOTL initiative. Changing Institutional Culture: One Example (cont.)

45 Changing Institutional Culture: Another Example An initiative of Craig Nelson, a faculty member committed to the campus SOTL initiative and a Carnegie Scholar, led to an institutional policy of partially funding travel of faculty members to attend teaching-related conferences. Candidates for support must show demonstrable connection to their scholarly interests in teaching and learning and provide a quid pro quo to the campus in return. Craig Nelson is a co-founder of the campus SOTL initiative. His contributions are many. Its very helpful to have a teacher-scholar of his prominence involved! Craig Nelson Biology to Craig

46 Changing Institutional Culture Further Examples Input from the Faculty Advisory Council and interested faculty researchers influenced the human subjects process to facilitate SOTL activities. Input from the Faculty Advisory Council and interested faculty researchers facilitated access to institutional data (registrar, etc.) for SOTL researchers. These achievements were important steps in developmental scaffolding for faculty embarking on SOTL projects. An aversive institutional review board decision regarding students as human subjects or inability to gain access to essential background data regarding the students in the study can be lethal to a SOTL project.

47 Campus Participation in SOTL Program of EventsSOTL Program of Events David Pace David Pace provides an overview of SOTL program impact.overview of SOTL program impact 14 main events in program; 11 in – about one every two weeks in fall and spring terms 108 campus units represented ( 74 academic departments/ programs)* Participants: 496 individual faculty members 286 tenure/tenure track (22% participation rate for T/TT) 51 individual staff 237 graduate students Total: 784 individuals (63% faculty, 30% graduate students, 7% staff)* Total attendance: 1598 (53/event in 1 st year and 77/event in 2 nd)* *Greatly exceeding expectations of many observers Assessing SOTL Program Impact: Participation Statistics

48 Selected Faculty Comments about the Campus SOTL Initiative We need SOTL The events I attended were terrific. Keep it up. And keep me in the loop. I was not able to participate directly in most SOTL events, due to numerous personal and professional obligations. But I was impressed by the degree to which SOTL spurred conversation on teaching among my colleagues. I also met with one SOTL participant to consult on his research project, and I was impressed with the enthusiasm he was bringing to his teaching and the related project. SOTL has been great for this campus (and, more importantly, our students), and I plan to participate enthusiastically next year. Thanks! SOTL is an outstanding source of intellectual support for the graduate school's efforts in "preparing future faculty." I realize that the program is in some sense intended to support and provide a forum for people producing the scholarship of teaching and learning. However, please do keep in mind how useful it can be to those of us simply working on teaching. I think you are doing a wonderful job, and I fully support your work SOTL is a very valuable asset to the university and the School of HPER. Please continue with your excellent efforts. I think you are beginning to make substantive improvements to the learning environment and are slowly raising awareness in the general faculty about the importance of teaching and how this may be approached through scholarship. I am afraid I have a rather dim view of "scholarship" about teaching or learning. Teachers should teach, learners should learn. Scholarship should be reserved for inquiry in genuine subjects. Scarce resources would be better devoted directly to teaching (e.g. smaller class sizes) or traditional substantive areas of research The ratio of positive to negative comments is actually much higher than in the sample here.

49 Support for SOTL by research establishment provided on same basis as research in the disciplines (matching funds, summer fellowships, etc.) Campus colleagues with specialized expertise voluntarily contribute time and effort to consult with scholars of teaching. Growing constellation of linked efforts affording opportunities and support (course portfolios, PFF, etc.) Large informal SOTL community on campus Samuel explains why multidisciplinary community is important to SOTL (beginning at 2 minutes: 10 seconds into the audio clip)Samuel explains why multidisciplinary community is important to SOTL Scholarship given birth under SOTL initiative begins to gain recognition (and invitations) beyond campus A visiting scholar of teaching and learning, Professor E. F. Redish, Department of Physics, University of Maryland, remarked that he had not previously experienced such a multidisciplinary community of scholars investigating issues of teaching and learning.Professor E. F. Redish After meeting members of this community, Lee Shulman, President of the Carnegie Foundation for the Advancement of Teaching, echoed similar sentiments... that this SOTL community has a truly extraordinary robustness and interdisciplinary character. Assessing SOTL Program Impact: Scaffolding

50 Think for a minute about your own campus. What administrative priorities or structures might help in your campus level SOTL program? What elements of the preceding example, if any, may be applicable? What hurdles might need to be overcome on your campus? Pause here to reflect on administrative features and obstacles on your own campus in the context of the Bloomington example. Press the button and share your thoughts. Your comments may prove useful in unforeseen ways. Besides, were REALLY interested! TASK D: Refining Your Own Campus SOTL Program or Plan

51 Other Campus Examples Elon College The Citadel Rockhurst University Abilene Christian University Notre Dame University Middlesex Community College Task E – Applicable Features Campus examples of SOTL initiatives from six different institutions of widely varying types are exhibited in the following frames. All of these campuses are participants in the AAHE Campus Program After surveying these examples, youll be asked to identify features of them that might be applicable to your campus program (Task E)

52 Elon College Multidisciplinary, Multiyear, $72,000 Investment –$6000 projects in each of 3 years – Projects directed by faculty-student research teams –Learning for BOTH student and teacher – Eight projects selected in years 1 and 2 Create intellectual engagement –New thinking in diverse fields –Application of learning to life – Opening spaces for reflective integration The magnitude of the Elon investment in the SOTL projects is notable for a school with about 4000 students. The projects focus on conditions of learning and the individual course is the unit of study. The use of student-teacher research teams to carry out each project is a distinctive feature. Elon jointly with Western Washington University won the first AAHE Going Public grant. John G. Sullivan, Powell Professor of Philosophy, kindly provided the information used in this tutorial. For more informationFor more information

53 The Citadel Mission –Increased campus awareness of and participation in SOTL Focus –Communication, Resources and Continuing Education –Self-selected research projects Highlights –Biweekly, participatory meetings with assignments Effectiveness –15% of full-time, tenure-track faculty at bi-weekly meetings –12% of full-time, tenure track faculty in classroom research Administrative Support –Attendance at functions –Financial support The Citadel SOTL initiative got a big boost from a Mimi Steadman workshop on classroom research. The Citadel SOTL website is rich in resources and ideas. Its well worth a browse. One distinctive feature is the schedule of assignments for the year given to participating faculty.Citadel SOTL website Professors Alix Darden and Suzanne Mabrouk spearhead work in this initiative. Suzanne can be e- mailed through the website. Alix was selected as a Carnegie Scholar for toAlix toAlix

54 Rockhurst University Beginnings –(Fall 1998) All University Symposium –(Spring 1999) Follow-up Symposium Year-long Carnegie faculty seminar –( ) Carnegie faculty seminar continued The Rockhurst Carnegie Seminar –Central Questions –Seminar Members –Discussions –Methods Products –Formal Letter on SOTL –Faculty SOTL Projects Selected Key Issues and Observations –Obstacles to Discussion –Interdisciplinary/Collaborative Approaches –Where's the beef? –Scholarly Teaching as best first path –To be a good consumer of the SOTL Rockhurst, a small (3000 students) independent university in Kansas City, conducted a year-long Carnegie seminar which resulted in a formal letter addressing reward for scholarly teaching and scholarship of teaching in tenure and promotion. An observation of the Rockhurst Carnegie Seminar … which is entirely consistent with observations at other institutions … is that a focus on scholarly teaching is a good first step to scholarship of teaching. Several faculty research projects are underway at Rockhurst. One researcher is Professor Anita Salem (mathematics), also a Carnegie Scholar of Teaching. Anita provided the information for this tutorial. to Anita to Anita

55 Abilene Christian University 19 Faculty Engaged in SOTL Projects Strong Institutional Support –Stipends for Materials & Resources Travel to Teaching-Related Conferences -Ongoing Peer Meetings,Videoconferences Abilene Christian provides an example of an effective SOTL initiative at very modest cost in a private institution with about 4500 students. Faculty participants make a year-long commitment to get their projects to the stage of publication. They are given small stipends for materials and resources. An outside consultant provides a day long workshop to help shape the projects. Ongoing support is provided through peer meetings and videoconferences with the consultant. Visit the ACU SOTL website or request a SOTL program brochure from Nancy Shankle, English Department Chair. Nancy steers the SOTL initiative at ACU. to NancyACU SOTL website to Nancy

56 University of Notre Dame Initial campus conversations with 90 campus leaders –SOTL needed support –RFP resulted in 9 funded SOTL projects Sample research question: Do new teaching methods in introductory engineering affect students learning? Support for SOTL teams includes these elements: –$5,000 per team for student time, equipment, supplies, faculty time –Consulting with methodology experts –Group meetings 2x/semester for mutual support –Help in dissemination of results The Notre Dame SOTL initiative features faculty research projects, campus conversations, and a Carnegie Scholar. Barbara Walvoord and her team of experts provide support to investigators in regional institutions as well as at Notre Dame to BarbaraNotre Dame SOTL initiative to Barbara Notre Dames approach – to develop a community of scholars and define SOTL by example – is similar to the approach in Bloomington. The impediments are similar too … like faculty reluctance to engage in a scholarship for which they feel they have no training.

57 Middlesex Community College Interdisciplinary group of 9-10 faculty Receive 1 course release for minimum 2-year commitment Meet biweekly for two-hour seminars Study SOTL - and how to create environs that foster intrinsic motivation Undertake individual faculty projects A goal at Middlesex is to link faculty projects to intrinsic motivation in students. Professor Donna Duffy (psychology), also a Carnegie Scholar, provided information about the Middlesex SOTL initiative. Donnas own project at Middlesex is Resilience as a Path to Integration in Abnormal Psychology. She chose resilience, a positive attribute, as a theme because abnormal psychology is generally about negatives … the problems people face. to Donna to Donna Whats common to all of the seven campus examples (including Indiana University)? -they involve institutional investment and support for the SOTL program -they are connected to AAHE/Carnegie national initiatives. -they involve faculty investigative projects into issues of teaching and learning

58 Task E: Campus Examples of SOTL Initiatives List features of the examples you have just seen that might be most applicable to your campus. Please reflect on the applicability of the campus examples just discussed to your own institution. Construct a mock-up of a SOTL program on your campus, drawing on the features of programs at other campuses. Press the button on the frame and send your model to us.

59 Unit 2B Faculty SOTL Projects Donetta Cothran – Kinesiology Valerie OLoughlin – Medical Sciences David Pace – History William Becker - Economics Rita Naremore – Speech and Hearing Sciences Dennis Jacobs – Chemistry Leah Savion – Philosophy Other Examples – Carnegie Scholars Task F: Minute Paper After viewing these projects, youll be asked to identify the most important points for you in the first four modules of the tutorial (Task F)

60 Donetta Cothran - Kinesiology For Donetta Cothran, a junior faculty member, interest in improving her students sense of fairness in grading of their papers led naturally into a scholarly project.scholarly project. Donetta gained insights about scholarship of teaching from her participation in a campus SOTL initiative and involvement in her own scholarly project.insights to Donetta Expectations and Effects of Graded Writing Assignments

61 Fostering Interactive Learning in a Large Science Course and Methodically Measuring the Effects Valerie Dean OLoughlin Medical Sciences A Neophytes Adventures in the Scholarship of Teaching Valeries innate interest in teaching found form and expression in the campus SOTL initiative. Her project is a Child of SOTL, conceived, born, and nurtured to scholarly product within the campus initiative. Scaffolding facilitated her acquisition of the concepts and resources needed to carry out her study. Valeries initial presentation of her work to peers was of interest to many both for the pedagogical techniques she employed and for her means of assessing their effects. Valerie describes the launch of her project Powerpoint presentation of the complete study to Valerie Details of Valerie OLoughlins study are considered later in the tutorial in Questions, Designs, and Resources.Questions, Designs, and Resources

62 What do I want my Students to be Able to Do? David Pace–History David Pace, a Carnegie Scholar of Teaching, describes his odyssey to the question at the heart of his scholarship; What are students supposed to be doing (or learning) in my course? He shares his realization that the above question led to understanding of his discipline in ways he never understood it before.odyssey to the question at the heart of his scholarship to David

63 Bill Beckers project is an example of a study that is not centered on ones own classes. Rather by studying selected published papers of others, he seeks to separate empirical results from conjecture about the student outcomes associated with classroom assessment techniques and other teaching strategies designed to engage students actively in the learning process. In this project, Bill also seeks to advance the scholarship of teaching and learning by establishing an eleven-point set of criteria which faculty members can use to evaluate inferential studies. Bills study culminated in a paper: What Does the Quantitative Research Literature Really Show about Teaching Methods?What Does the Quantitative Research Literature Really Show about Teaching Methods? to Bill William Becker Economics What Does the Quantitative Research Literature Really Show about Teaching Methods?

64 The problem: a group of faculty members concerned that teaching excellence was getting short shrift in the faculty evaluation process The initial question: What can (and do) student evaluations really tell us? Rita Naremore Speech and Hearing Sciences The department had a culture that valued excellent teaching, and in some ways took it for granted. Salary increases were based on data provided using the campus faculty annual report form It had become abundantly clear that those faculty members engaging in innovative or particularly effective pedagogical change were not being rewarded because nobody knew what they were doing or how effective it might be.campus faculty annual report form Rita introduces the project Powerpoint presentation of complete study to Rita A Departmental Level SOTL Project

65 Peer evaluation presented the group with a much larger problem. After much heated discussion, it became clear that the occasional visit to a classroom by ones colleagues did not seem an adequate form of peer evaluation. This faculty agreed that what was missing was a systematic review of course organization and materials (including exams, visual support for class presentations, etc.) conducted by people outside the department who knew the content. Once the question regarding student evaluations seemed answered, attention turned to peer evaluation. The faculty consensus: traditional classroom visitation is not very revealing of anything useful. What we really need is evaluation of course structure and materials used in the class for teaching and evaluation. Wed like to have periodic external peer review of these materials. A Departmental Level SOTL Project Issue # 2: Peer Evaluation

66 The biggest concern: IT TAKES TOO LONG TO PUT ALL THIS STUFF TOGETHER! What are we doing? Were working toward the development of course portfolios, to be done over time and turned in by faculty members every second or third year, not every year. Although agreement in principle was quickly attained, agreement in practice was not. Several faculty members remained convinced that they did not have time to put together their teaching materials in a way that made them accessible to an outside reviewer, especially if they had to explain what they were trying to do. Nonetheless, the department is continuing to work toward a system of course portfolios, along with a checklist that can be used by outside reviewers who are looking at the portfolios. A Departmental Level SOTL Project: Ongoing Concerns with Peer Evaluation

67 A Departmental Level SOTL Project: The Campus and National Ties Review the Relationship Review the Relationship of Peer Review of Teaching, Course Portfolios, and SOTL Visit the Indiana Campus Course Portfolio Initiative More Information About the Consortium Project Led by Nebraska to Simon The connection to course portfolio projects exemplifies the way SOTL may be synergistically connected to and integrated with other teaching-related initiatives. Simon Brassell Geological Sciences Chair of the Campus Course Portfolio Initiative The departmental course portfolio effort should eventually tie in with a campus- wide effort now underway, conducted by an interdisciplinary team in conjunction with a national project funded by Pew, based at the University of Nebraska, and involving four other research universities.

68 An Alternative Approach to General Chemistry Assessing the Needs of At-Risk Students with Cooperative Learning Strategies Dennis Jacobs Professor of Chemistry University of Notre Dame The next 7 frames contain details of this project. The project exemplifies further the tie between SOTL and course portfolios as well as providing an excellent example of each! Dennis Jacobs is a Carnegie Scholar of Teaching and Learning. The project in complete detail The project in complete detail (log-in required) Skip to the next SOTL projectSkip to the next SOTL project. to Dennis An Exemplary Course Portfolio And A Superb Model of SOTL

69 - 4 - Rationale: Recognized Problems: At-risk students (Math SAT 630): dropped out of General Chemistry. didnt take any advanced science. frustrated by large lecture format. Alternate Course Design: Similar requirements and lectures. Comparable exams. Various activities involving structured cooperative learning. Initial Comments on Proposal Only delaying inevitable failure. Efforts should be focused on the best not the at-risk students. Comments that colleagues initially made to Dennis are shown in the last bullets. Dennis proved them wrong. He demonstrated convincingly that at risk students could perform at the same standards as other students in general chemistry and go on to succeed in organic chemistry (a sequel course) at the same rate. An Alternative Approach to General Chemistry Assessing the Needs of At-Risk Students with Cooperative Learning Strategies Dennis Jacobs (University of Notre Dame)

70 - 5 - Documentation: At-risk students markedly more likely to drop or fail the course in the traditional class format. As you view this project,, note Dennis methodologies and the thoroughness of assessment. This is a well designed deed of scholarship. Ask yourself at the end, whether you feel convinced from this brief slide series of the impact of his alternative course for at risk students. An Alternative Approach to General Chemistry Assessing the Needs of At-Risk Students with Cooperative Learning Strategies Dennis Jacobs (University of Notre Dame)

71 Implementation: Aim: Provide improved learning opportunities for at-risk students. Develop more effective teaching in large lecture format. Alternate Course Design: Introduced opportunities for structured cooperative learning including: discussion of concepts in pairs. small group in recitation sections. work as pairs in laboratory. Mandatory recitation sections: more time committed to class. direct contact with instructors An Alternative Approach to General Chemistry Assessing the Needs of At-Risk Students with Cooperative Learning Strategies Dennis Jacobs (University of Notre Dame)

72 Traditional vs. Alternative Classes: Similarities: Size (250 students), text, chapters. Lecture time (3 hr), lab time (2.5 hr). Lecture format (Powerpoint slides and demonstrations). Exam format and many exam questions. Differences in Alternative Section: Mandatory recitations (1 hr/wk, 20 students); attendance 95% vs. 10%. Weekly homework (10 vs. 30, graded). On-line quizzes (www chapter reviews). Weekly feedback from homework, group problems, on-line quiz, in-class questions. Personal contact with instructor and follow-up if performance declined An Alternative Approach to General Chemistry Assessing the Needs of At-Risk Students with Cooperative Learning Strategies Dennis Jacobs (University of Notre Dame)

73 Impact of Alternative Section: Assessment Strategies: Effects on conceptual understanding, problem-solving and self- confidence: feedback from students. evaluation of individual elements of the cooperative learning activities. Immediate and long-term benefits: retention of at-risk students. success in advanced science classes. Data Collection: Recording in-class learning activities. Tracking individual grades and progress. Longitudinal study of at-risk students: progress in subsequent classes An Alternative Approach to General Chemistry Assessing the Needs of At-Risk Students with Cooperative Learning Strategies Dennis Jacobs (University of Notre Dame)

74 Measurement of Impact: Success of At-risk Students: Better grades in General Chemistry. Improved retention in class. Higher success rate in subsequent classes An Alternative Approach to General Chemistry Assessing the Needs of At-Risk Students with Cooperative Learning Strategies Dennis Jacobs (University of Notre Dame)

75 Longitudinal Impact (follow-on course): Intervention introduced between 1996 and 1997 Organic chemistry is a follow–on course to general chemistry. In 1997, a higher percentage of at risk students who had taken Dennis alternative general chemistry course completed organic chemistry than the percentage of students in the traditional course. An Alternative Approach to General Chemistry Assessing the Needs of At-Risk Students with Cooperative Learning Strategies Dennis Jacobs (University of Notre Dame)

76 Investigating Pet Theories and Naïve Misconceptions Leah Savions personal interest in learners cognition and interference effects of naïve misconceptions caught on with faculty colleagues (the issue crosses most disciplines) and led to a cross-disciplinary study. Leah explains misconceptions and their effects A brief overview may be gained from a one page summary of the projectone page summary of the project Add student misconceptions from your own experience to Leahs Incomplete Inventory of Misconceptions Incomplete Inventory of Misconceptions Further detail may be gained from her working paperworking paper Leah warmly invites your peer review and comment ( to Leah)peer review and comment What force acts on a coin that has been tossed up in the air? The large majority of students who completed a course in mechanics in MIT gave the same wrong answer as the totally untrained students, citing the original upward force of the hand. The pet theory of motion at work here is that theres no motion without force. Pet theories are involuntary explanatory constructs that we all build from a very early age, in an attempt to understand the world around us, to build causal connections between events, and to enhance our sense of control over the environment. These theories are about every aspect of life. They are amazingly universal, are based on surface features of the relevant data, fragmented, inconsistent, contain principles that emerge spontaneously, complex, intricate, serviceable, and seemingly well organized. These robust theories are not normally tested against scientific, social or logical facts. A specialized psychological architecture that contains principles of quick reasoning, rapid accumulation of information, and economy of cognitive operations, may underlie and guide their construction and their perseverance in face of contradictory evidence. Pet theories inevitably imply what I termed here naïve misconceptions, that prove extremely resistant to change. When the naïve meets the academically acceptable theories we deliver in class, the better students either attempt to alter the new information, or combine the incompatible principles, or, most commonly, adapt the new theory as a school bound explanation, not applicable in everyday life. Leah Savion Philosophy

77 Scholarly Projects of Faculty (Continued) -Framing the Question -- Context of the Project -- Gathering the Evidence -- Emergent Findings -- Conditions for doing SOTL -- Benefits of the Work -- Lessons Learned Opening Lines may be a useful source for identifying issues of teaching and learning in ones practice and framing them. Many faculty will be able to identify with the thoughts and experiences of the authors of these case studies. For faculty members who feel that they do their thinking about teaching in isolation within their departments or institutions, this publication may feel like a conversation with colleagues. Where you can learn about additional scholarly projects. Opening Lines contains 8 case studies of approaches to individual projects by Carnegie Scholars. The seven bullets below form the outline of the narrative in each case study. Style and tone are similar to those in the video vignettes used in this tutorial. Opening Lines comes with a CD containing a variety of tools and resources for scholarly projects. These include interviews (some as video clips), survey instruments, protocols, and reference sources. The publication showcases recent work in CASTL CASTL

78 "MINUTE PAPER" 1. What were the 3 most important points presented in Units 1A, 1B, 2A and 2B of this tutorial? 2. What 3 things would you most like to learn in the remaining two modules, Units 3A and 3B, of the tutorial?. This Classroom Assessment Technique (CAT) denotes the end of Unit 2B. The formative feedback offered by you to us here will help make this a better tutorial for everyone. Please press the button and send your feedback. If youve worked through the tutorial continuously to this point, you might want to take a break before entering Unit 3 - the last stretch of the continuum from alpha to omega (and its more than a third)! continuum from alpha to omega Task F: Looking Back Over Travel Along the Continuum Thus Far

79 Unit 3A Bridges to Productivity How Could I Do Scholarship of Teaching and Learning (Genres of SOTL) Task G: Reflecting on Genres Approaches to Scholarship Via Classroom Research Projects Where to Publish and Present Sources of External Funding Task H: Reflecting on Classroom Research and Key Reference Books Unit 3 may be thought of as a rudimentary tool kit for embarking on scholarly projects in issues of teaching and learning. This is the longest unit. 3A surveys some general considerations. 3B focuses on project design.

80 Still Another Look at What We Mean by the Scholarship of Teaching and Learning Before venturing into a mélange of categorization schemes, it may be well to pause for another overview of this scholarship. Do the following situations represent scholarship of teaching and learning? 1.A faculty member reforms a course, building into the reform all three criteria of scholarly teaching. She puts the course on-line not only for students but for all faculty in the discipline, and furthermore invites faculty to peer review and critique it on the web. The course catches the attention of faculty at another institution. They are interested in it and invite the course author to come there and give a colloquium on the course. The author does so and reflects on the attributes of the course for her students. Subsequently, the department in the second institution adopts large elements of the authors course to improve their own undergraduate program. 2. An assistant professor in quest of tenure develops a course portfolio as evidence of his effectiveness in improving student learning. The promotion and tenure committee routinely sends candidate packages to outside reviewers. External reviewers comment on the excellence of this portfolio as convincing evidence of the candidates teaching effectiveness. The candidate is granted tenure. The course portfolio is subsequently disseminated through the state university system as a model for other faculty.

81 Still Another Look at What We Mean by the Scholarship of Teaching and Learning (Cont.) If you accept Shulmans quasi-definition, you will probably conclude that both situations in the previous frame DO represent scholarship because they satisfy all three of his conditions. As key reformers urged, the concept of scholarship is broadened to include useful contributions that do not necessarily conform to conventional scholarly forms in the disciplines. To be sure, scholarship of teaching and learning includes publication in journals, but it also includes other forms, as in the two previous examples, that contribute to the advancement of teaching and learning both directly and indirectly.Shulmans quasi-definition As key reformers urged To survey possibilities for this scholarship, we exhibit several categorization schemes. First, well consider Craig Nelsons Genres of SOTL. Then well look at traditional and classroom research and qualitative and quantitative methods. All of these schemes are messy and inconsistent. The genres, for example, sometimes seem separated by unit of analysis at other times by type of research design. And as we see from the two examples in the preceding frame, they are not necessarily inclusive of all possible forms of scholarship. Similarly, much good scholarship combines elements of traditional and classroom research as well as qualitative and quantitative methods. Well, why do we even bother with the classification schemes if they are inconsistent? Because they give us valuable points of possibility. They may be thought of as topographical features that map the territory. The actual point at which one locates ones individual deed of scholarship is usually somewhere between the points of possibility.

82 Different Genres of SOTL –Reports on Particular Classes –Reflections on Years of Teaching –Larger ContextsComparisons –Formal Research –Meta-Analyses Genres of SOTL overlap and can be combined or subdivided variously. The particular examples of scholarship for each genre are chosen to illustrate importance for improving teaching and learning and largely excerpted from Craig Nelsons paper on genres: "How could I do scholarship of teaching and learning?" "How could I do scholarship of teaching and learning?" Two opening points: 1. Learning and teaching are complex activities where approximate, suggestive knowledge can be very helpful, and, indeed, may often be the only kind that is practical or possible 2. Much important expertise on teaching resides in the day to day practices of good faculty. Typically, this knowledge remains private and is totally lost when its possessor retires. A key task in this field is systematically making much more of this expertise public. Craig Nelson introduces the genres of SOTL How Could I do Scholarship of Teaching and Learning?

83 Reports on Particular Classes It Worked! Important pieces of our expert knowledge as experienced practitioners can be preserved by writing up exemplary approaches to content or pedagogy that work especially well in our own classes. In this genre, the teacher's own impressions of the effectiveness frequently serve as sufficient assessment. The trend now is to try to document the effectiveness a bit more formally using CATs and CR. Before & After: Assessments of Changes in Practice -Qualitative The many examples of this genre in Angelo and Cross include a calculus class (pp ) in which the professor wanted to help students improve their problem solving skills. This example illustrates the process of refining the pedagogical questions and the successive modifications that are often necessary to make new pedagogical approaches work successfully. In this case, the new pedagogy improved student success sufficiently that no student made an F, despite the maintenance of high academic standardsAngelo and Cross -Quantitative Group study approaches to calculus decreased the D/F/W rate for African Americans from 60% to 4%. See R. E. Fullilove & P. U. Treisman Mathematics Achievement Among African American Undergraduates at the University of California, Berkeley: An Evaluation of the Mathematics Workshop Program. Journal of Negro Education 59: How Could I do Scholarship of Teaching and Learning? Different Genres of SOTL –Reports on Particular Classes –Reflections on Years of Teaching –Larger ContextsComparisons –Formal Research –Meta-Analyses

84 Reflections on Years of Teaching Essays Developing Good Ideas An example providing good articulation of the rationale for SOTL is L. S. Shulman Teaching as community property: Putting an end to pedagogical solitude. Change 25: 6-7. Summaries of Expert Knowledge Gained by Self-Reflection and Experimentation Examples are P. Frederick The Dreaded Discussion: Ten Ways To Start. Improving College & University Teaching 29: P. J. Frederick The Lively Lecture--Eight Variations. College Teaching 34:43-50 Many course portfolios fit here too. Path breaking examples can be accessed through links at Implementing Entities: Carnegie and Pew. Another example by Randy Bass is ( Pew Integration of Larger Frameworks with Classroom and Curriculum Practice J. D. Herron Piaget for Chemists: Explaining What "Good" Students Cannot Understand. Journal Chemical Education 52: One factor that explains why bright, hard-working students can do poorly and how we can help them. Easily applicable in all quantitative fields. R. J. Kloss A nudge is best: Helping students through the Perry scheme of intellectual development. College Teaching 42: Another factor that explains why bright, hard-working students can do poorly and how we can help them. Easily applicable across the curriculum. How Could I do Scholarship of Teaching and Learning? Different Genres of SOTL –Reports on Particular Classes –Reflections on Years of Teaching –Larger ContextsComparisons –Formal Research –Meta-Analyses

85 Larger Contexts: Comparisons Across Courses and Student Change Over Time Qualitative Studies to Explore a key Issue Wm. G. Perry, Jr. [1970] Forms of Intellectual and Ethical Development in the College Years, A Scheme. New introduction by Lee Knefelkamp. Jossey-Bass. The impetus here was the observation that students could flunk out of Harvard despite working quite hard at learning the course material. The longitudinal design used extensive interviews with students at the end of each of their four undergraduate years. Patterns of intellectual development were inferred and checked for inter-judge reliability. A very influential study. Quantitative comparisons of Different Courses or Sections M. D. Sundberg & M. L. Dini Science majors vs nonmajors: Is there a difference? Journal of College Science Teaching. Mar/Apr 1993: Question: Does covering more teach more?. Both courses taught with traditional pedagogy and by multiple instructors, but with different intensities of 'coverage.' Learning assessed with the ACT exam for AP Biology "The most surprising, in fact shocking, result of our study was that the majors completing their course did not perform significantly better than the corresponding cohort of nonmajors." Comparisons over an Array of Courses with a Common Assessment Instrument R. R. Hake Interactive-engagement vs traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses. American Journal of Physics 66: ( Uses a common test of Newtonian physics to compare increases in understanding achieved by a wide array of pedagogies in introductory physics courses at institutions ranging from high-schools to Harvard. Found that "interactive engagement" approximately doubles the amount of physics earned. An especially important model for emulation in other disciplines. How Could I do Scholarship of Teaching and Learning? Different Genres of SOTL –Reports on Particular Classes –Reflections on Years of Teaching –Larger ContextsComparisons –Formal Research –Meta-Analyses

86 Formal Research How Could I do Scholarship of Teaching and Learning? Different Genres of SOTL –Reports on Particular Classes –Reflections on Years of Teaching –Larger ContextsComparisons –Formal Research –Meta-Analyses Experimental Analyses C. M. Steele A threat in the air: How stereotypes shape intellectual identity and performance. American Psychologist 52: Classroom Incivilities R. Boice Seminal study on incivilities. Identifies and counts perceived incivilities committed by faculty as well as students and investigates relation between them. Journal of Research in Higher Education 37: The Impact of One Minute Papers on Learning in an Introductory Accounting Course E. D. Almer, K. Jones, & C. Moeckel Four hypotheses investigated simultaneously in one class using a fractional factorial design. Issues in Accounting Education 13: The One-Minute Paper: Some Empirical Findings J.F. Chizmar & A. L. Ostrosky Four teachers each taught control and experimental sections. Data were analyzed using a multiple regression model. Journal of Economic Education 29:3-10 This study like the one above on incivilities could also be categorized under Larger Contexts The last two studies of one-minute papers are good examples of formal research at the classroom teacher level. They are also important contributions to the structure of evidence that one-minute papers improve student learning.Journal of Economic Education 29:3-10Larger Contexts What constitutes formal? Does formal reside in the lower right quadrant of Rices diagram? Not necessarily. We use formal research to mean forms of scholarship in which the principal goal is research. Usually such scholarship will incorporate one or more features of conventional research paradigms. Formal research can still be classroom-based and related to active practice (see the last three examples below).Rices diagram

87 Meta-Analyses Annotated Bibliographies R. N. Johnson, D. M. Enerson & K. M. Plank Diversity: A Selected and Annotated Bibliography Center for Excellence in Learning and Teaching. Pennsylvania State University.Diversity: A Selected and Annotated Bibliography Brief Annotated Summaries of Key Research Findings T. A. Angelo The campus as learning community: Seven promising shifts and seven powerful levers. AAHE Bulletin 49:3-6. R. B. Barr & J. Tagg From teaching to learning: A new paradigm for undergraduate education. Change 27: Formal (Quantitative) Meta-Analyses L. Springer, M.E. Stanne & S.S. Donovan Effects Of Small-Group Learning On Undergraduates In Science, Mathematics, Engineering And Technology, A Meta-Analysis. National Institute for Science Education, University of Wisconsin. 608/ [average effect size "would move a student from the 50th percentile to the 70th..."] How Could I do Scholarship of Teaching and Learning? Different Genres of SOTL –Reports on Particular Classes –Reflections on Years of Teaching –Larger ContextsComparisons –Formal Research –Meta-Analyses

88 RESPONDING TO THIS TASK IS NOT JUST BUSYWORK! Reflection on the points of possibility described in the genres, may be an important step in conceptualizing your own work in SOTL. You may want to read one or more of the example papers before attempting this task. Your conception of your own SOTL work need not coincide with one of the genres, of course! Where does the course portfolio fit in the genres described?the genres described Reflecting on the genres, which seems most appropriate for a scholarly project of your own? Which seems easiest for you to do? Are there forms of SOTL for which the genres described do not apply? If so, how would you name one or more additional genres?the genres described Task G: Reflecting on Genres

89 Classroom Research Distinguishing classroom research from traditional research A grand schematic of classroom research Classroom Assessment Techniques Effective Grading The Course Portfolio Task H: Reflecting on Classroom Research Classroom research is described. A schematic diagram is exhibited and used to show strengths of various useful books. Finally, in Task H you will be asked to identify the step in classroom research that seems easiest and the reference book that seems most useful. Approaches to Scholarship Via Classroom Research

90 Classroom Research is not traditional research conducted in or on classrooms. It is a specific methodology designed for discipline oriented teachers without training or experience in the methods of educational research. Classroom Research is ongoing and cumulative intellectual inquiry by classroom teachers into the nature of teaching and learning in their own classrooms. Inquiry into a question about how students learn typically leads to new questions and thus to continual investigations through classroom research. Cross & Steadman Finding that almost no relationship existed between research on learning and collegiate teaching practices (professors were either oblivious to such research or ignored it), K. Patricia Cross concluded that the research itself was at fault for failure to pay attention to actual classrooms. She argued that teaching and learning reforms could only occur if they were based on concrete classroom situations. Cross made how to do it her calling card. M. Lazerson, U Wagener, and N. Shumanis. Teaching and Learning in Higher Education. p. 17 Change 32: Many journals that publish articles about teaching and learning, both within disciplines and across disciplines, invite articles based on classroom research. The term action research is sometimes used synonymously with classroom research. The former dates from the work of Kurt Lewin in the 1940s and may be defined as Research carried out by practitioners with a view to improving their professional practice and understanding it better. Borg, Gall & Gall Applying Educational Research 3 rd ed. Longman,: 390 Cross, K.P. and Steadman, M Classroom Research: Implementing the Scholarship of Teaching, Jossey-Bass Classroom Research

91 TraditionalClassroom OriginState of present educational literatureProfessors teaching practice PurposeContribute to existing state of educational theory Obtain practical knowledge applicable in limited circumstances RequirementSpecialized training / broad grasp of pertinent literature Specialized training not essential but developmental scaffolding may be BenefitField and researcherStudents and professor Traditional and Classroom research are compared for the sake of characterizing two points of possibility. One is not advocated over the other and its an oversimplification to view them as independent. Traditional research is often performed in classrooms. Classroom research may be enhanced by traditional methods and measures, especially if the results are for publication in national journals. For example, use of an off the shelf assessment instrument with widely accepted validity may be preferable to a home grown test or survey with unknown validity. Using someone other than the person who determines grades to collect potentially sensitive data from students is also a traditional method that may enhance a classroom research effort (and make it compliant with the local human subjects policy!) A key point is that classroom research is a good point of embarkation for the typical faculty member using materials at hand and without specialized training in educational research methods. Classroom research is usually directed at local teaching issues or questions of specific interest to the teacher. Classroom research is not about finding universal truth but rather contextualized truth. Tom Angelo Traditional and Classroom Research

92 Improving Teaching and Learning via Classroom Research Improved Teaching and Learning Existing Scholarship Recorded Observation Summative Assessment Formative Assessment Institutional Information Base Issue Goal Reflection Analysis IMPETUS INFORMATION GATHERING EFFECT SCHOLARLY PRODUCTIVITY Synthesis APPLICATION Our grand schematic of classroom research! Schema with similar features are common; We envision three major components to a research project; a motivating goal or issue, acquisition of information bearing on achievement of the goal or resolution of the issue, and effects identifiable from the juxtaposition of impetus and information. Sometimes the project involves an intervention, but not necessarily. Schema with similar features We argue that if the scholarly project is done well, teaching and learning will inevitably improve regardless of the results of the study. Why so? Because the classroom research by its very nature will at least improve attention to students and information they provide, thereby improving understanding and communication. Student performance is also positively motivated by evidence that teachers care enough to study them and the efficacy of their learning. After the third major component in the above schema (Effect) the possibility of a scholarly product may emerge (curriculum materials, article, presentation, etc.) Such a product may enhance the institutions information base as well as the existing body of scholarship.

93 Classroom Assessment Classroom assessment is systematic and formative Class is the unit of measurement rather than the individual Conditions of learning may be assessed rather than student performance. Correct and incorrect are not the emphasis. Unexpected rather than expected responses are often most useful. Review Unit I material about classroom assessment/research Angelo, T. & Cross, K.P. (1993) Classroom Assessment Techniques: A Handbook for College Teachers (2nd Ed) Jossey-Bass Improved Teaching and Learning Existing Scholarship Recorded Observation Summative Assessment Formative Assessment Institutional Information Base Issue Goal Reflection Analysis IMPETUS INFORMATION GATHERING EFFECT SCHOLARLY PRODUCTIVITY Synthesis APPLICATION Defining features of classroom assessment are shown in the bullets.They are quite different from the features of summative assessment associated with traditional grading. The book belowhelps faculty clarify their teaching goals, explains formative assessment to improve achievement of goals and provides a catalog of 50 classroom assessment techniques (CATs).

94 Effective Grading Primary Trait Analysis (PTA)… building scales that make performance criteria explicit in order to: -Categorize/classify student work. -Benchmark student learning and document changes. -Improve validity of grading. Walvoord, B. and Anderson, V. (1998) Effective Grading: A Tool for Learning and Assessment, Jossey-Bass. Improved Teaching and Learning Existing Scholarship Recorded Observation Summative Assessment Formative Assessment Institutional Information Base Issue Goal Reflection Analysis IMPETUS INFORMATION GATHERING EFFECT SCHOLARLY PRODUCTIVITY Synthesis APPLICATION The book below: Is a useful resource for classroom researchers Aims at documenting how well students are achieving cognitive objectives and at providing evidence for change in achievement over time Treats almost any conventional objection or concern about grading

95 I was familiar with teaching portfolios … but thinking about teaching as scholarly inquiry began to lead me in the direction of something I had not seen anyone else doing: a portfolio that focused on the course rather than on all of ones teaching. Being a social scientist, I began to think of each course … as a kind of laboratory - not a truly controlled experiment of course but as a setting in which you start out with goals for student learning, then you adopt teaching practices that you think will accomplish these and along the way you can watch and see if your practices are helping to accomplish your goals, collecting evidence about effects and impact. W. Cerbin quoted in Hutchings, P. (Ed.) 1998 The Course Portfolio, AAHE As a kind of laboratory notebook for recording data and observations over time, the course portfolio keeps the teachers experience from being lost with the teachers memory. Its a possible means of capturing wisdom of practice. Review Unit I material on course portfolios Review a course portfolio exampleReview a course portfolio example. The Course Portfolio

96 Where to Publish and Present Venues for publication and presentation of work in SOTL abound. Journals and conferences may be discipline-based or inclusive of all disciplines, refereed or non-refereed, and local, regional, national, or international. Publications may be printed or on-line and conferences may be physical or virtual (teleconference). Many sources of publication are evident in references given throughout the tutorial. Additionally, over 100 journals publishing SOTL are accessible through a list of journals publishing scholarship of teaching and learning.list of journals publishing scholarship of teaching and learning Opportunities to present in cross-disciplinary venues (calls for proposals and conference announcements) are provided by the Professional and Organizational Development (POD) Network in Higher Education and the American Educational Research Association (Division J- Postsecondary Education). Both POD and AERA maintain electronic mail lists (listservers) which include calls for proposals and conference announcements from all over the world. More information about these lists are available at the above links.Professional and Organizational Development (POD) Network American Educational Research Association (Division J- Postsecondary Education).

97 Potential Sources of External Funding Pew Charitable Trusts Fund for Improvement of Postsecondary Education National Science Foundation Woodrow Wilson National Fellowship Foundation Alfred P. Sloan foundation Department of Education Spencer Foundation Andrew W. Mellon Foundation Eli Lilly Foundation Sources of grant funding for specific topics and categories in higher education are generated by search engines operating on SPIN and COS databases. Most institutions subscribe to at least one of these databases. See your local sponsored research services or institutional research services for more information. Some sources, but by no means an exhaustive list, are shown below.

98 Task H: Reflecting on Classroom Research Which major step in the classroom research schematic (determining the goal or issue, gathering essential information, or determining effect) seems the easiest for you to carry out? Why?classroom research schematic Which of the texts described in this section seems most immediately useful to you?described in this section

99 Unit 3B Questions, Designs, and Methods Framing Questions Task J: Examining Valeries Questions Task K: Framing Your Own Question Designs for Studies Choosing Measures Guiding Questions in Choosing Methodology Task L: Designing Your Own Project Summary of Standards Task M: Evaluation of the Tutorial We begin the final module of the tutorial with a disclaimer; the possibilities are too many for adequate survey here. The topics in this module could easily fill a textbook (and, in fact, do fill many textbooks!) We limit coverage to a few possibilities.

100 Scholarship of teaching and learning does not necessarily begin with a question. To record what worked, the wisdom of experience or to describe observed phenomena without attempting intervention can constitute scholarship. To integrate the work of others, as in constructing a continuum of SOTL experience from Alpha to Omega, or as in editing a collection of papers into a book might also constitute scholarship. Nevertheless, much scholarship is rooted in inquiry because research is rooted in inquiry. Faculty members do not necessarily receive formal preparation for teaching, much less scholarship of teaching. Therefore, to expect faculty members to know how to transform their issues and goals of teaching and learning into researchable questions without a little tutoring does not seem quite reasonable. As many dissertation advisors and doctoral students will attest, the most difficult step in the whole research process is often framing the question to be addressed. A well framed question is one for which procedures can be devised that offer the possibility of arriving at an answer. The next series of frames are rather narrowly focused on this particular challenge. Framing the Question

101 1.Define a goal 2.Ask and answer questions to refine the goal (Colleagues are usually helpful.) 3.Create a succinct summary of a specified goal 4.Ask what evidence would reveal that the goal is achieved? 5.Frame possible researchable questions Teachers commonly think of goals in terms of improved learning outcomes for students. Often, a study is built upon one or more interventions to achieve a goal and the question becomes Does the intervention lead to achievement of the goal? The goal approach also includes inquiries which Pat Hutchings might characterize as What works? However, a study based on a goal approach need not involve an intervention. A well-framed question could simply address whether an intended goal is presently being achieved or whether a goal is even feasible to achieve. Framing Questions (Goal Approach)

102 Framing Questions (Issue Approach) Criteria for selection of issues Investigable (not necessarily empirical) Bounded and well-defined Significant (not necessarily statistically) Considerations for investigation of issues Length of time needed Complexity of procedures Availability of subjects Availability of support (resources, personnel, funds) Questions can involve investigation of issues rather than achievement of goals; e.g. How do students who do not meet prerequisites fare compared to those who do? In Opening Lines, Mills Kelly describes how his department chair framed Mills question by asking, How do you know that using the Web as opposed to depending on paper (the way most of us have taught history) is transforming student learning – and, if so, whether for good or for ill? Other types of questions like What does the current learning environment for students in my course look like to them?, How do they perceive my instruction? or How do students who yield evidence of deep understanding in my course, gain that level of understanding? can lead to investigative studies.Opening Lines

103 Making Vague Questions Answerable – Using Operational Definitions Less framed Do students who help others learn an academic discipline learn it better themselves? Do students learn more in small classes? What is the optimum number of homework assignments to give in a beginning math class? More framed Do students in CMSC 250 who tutor students in CMSC 150 perform better on the CMSC 250 final exam than students who do not tutor but have similar grades in CMSC 150? Do students in sections of Phys 118 enrolling fewer than 50 students perform better on the departmental final exam than students from sections enrolling more than 75 students? Do students enrolled in M036 who are given a homework assignment every week perform differently on the departmental final exam than students enrolled in M036 who are given homework every class period? Vague questions are made answerable by using operational definitions of terms to improve conciseness. Operational definitions sharpen the question and afford possibility of clear answers. The price paid is usually a sacrifice in generality of answers obtained. Considering small classes to be less than 50 students and learning to be characterized by final exam performance as in the above example are arguable definitions, of course. Nevertheless these operational definitions afford obvious and necessary clarity to the structure of the study.

104 Valerie Dean OLoughlin Medical Sciences A Neophytes Adventures in the Scholarship of Teaching Review the introduction of Valeries project Valeries Questions 1.Will the use of interactive learning activities in A215 lecture improve exam performance? 2.Will these activities improve lecture attendance and create a more active student engagement in lecture? 3.Will these activities improve overall merit success rate in A215? Task J: Examining Valeries Questions Valeries questions need further specification. What terms, in particular, need improved operational definition? After submitting your answers, compare them with the answers in the following frame.

105 A Neophytes Adventures in the Scholarship of Teaching Task J (Answers) Terms particularly needing improved operational definition: interactive learning activities Primary intervention: use of one or more interactive learning activities per lecture session Examples of Learning Activities * Memory Matrices Learning Exercises Sample Exam Questions Muddiest Point active student engagement Valerie defines this term orally (beginning at 1 minute: 40 seconds into the video clip) Valerie defines this term orally merit success rate merit success rate = total A,B,Cs initial enrollment * Angelo, T and KP Cross (1993). Classroom Assessment Techniques: A Handbook for College Teachers. 2 nd Ed. San Francisco: Jossey-Bass

106 Framing Questions as Hypotheses H1: Students who write one-minute papers will perform better on a subsequent a. essay b. multiple choice quiz than students who do not write one- minute papers. H2: Students who address their one- minute papers to a novice audience will perform better on a subsequent quiz than students who address their papers to the instructor. H3: Students whose one-minute papers are graded will perform better on a subsequent quiz than students whose one-minute papers are not graded. H4: The difference in quiz performance between students who do and do not write one minute papers will be less for higher ability students than for lower ability students. Almer, E.D., Jones, K., & Moeckel, C.L. (1998) The Impact of One- Minute Papers on Learning in an Introductory Accounting Course. Issues in Accounting Education (13) 3 H1 actually contains two hypotheses. Why? In some studies, questions are most usefully framed as hypotheses to be tested. All of these hypotheses were successfully tested in the referenced study.

107 Task K: Framing Your Question 1.Write a tentative question to be addressed in one of the courses you teach. 2. Discuss your question with one or two colleagues for the purpose of framing it in the clearest and most answerable way. Encourage your colleagues to challenge your framing. 3.Write your well-framed question at the end of this process. THIS TASK MAY TAKE YOU AWHILE. You might want to stop here, think about your question, take it down the hall for discussion with colleagues, and bring back the results after lunch. You might also want to compare and contrast it to the questions in Faculty SOTL Projects before writing it in well-framed form and pressing the button to send it.Faculty SOTL Projects

108 What is a Design for a Study? A plan or protocol for carrying out the study An underlying scheme that governs functioning, developing, or unfolding The design should include a timetable for significant milestones, specification of measures to be used in acquiring information, and provision for obtaining needed resources (Example of milestone: institutional approval to use your students as subjects. Two examples of needed resources: grade distributions of students in the target course over the last several years; a colleague to conduct interviews of your students.) It may be necessary to modify the design as work progresses. In general, the greater the degree of qualitative information gathering, the more likely is modification in the design as the study progresses. Its important that any design modification be the result of careful consideration rather than something that just happens in the course of events without the implications for the goals of the project having been thought through. A good design always promotes efficient and successful gathering and analysis of the needed information. Tthe design makes clear what information is being gathered (Example: student perception by means of fixed interview procedure in focus groups) or is being measured (Example: student learning by means of 3 exams). The design also makes clear what, if anything, is being manipulated. (Example: the study is designed so that some students attend review sessions before each exam while others do not.) The nature and frequency of measurement and manipulation are made clear along with factors that the investigator is attempting to take into account because they may explain the result even though they are not the focus of the study. (Examples of such factors are students class attendance or backgrounds.)

109 A Qualitative or a Quantitative Study? Danger: This may not be the best question to ask! Why may the above question not be the best one to ask? Because it often tends to polarize faculty along their disciplinary lines and this may not be in the best interest of excellent scholarship of teaching and learning! No factor is more influential in shaping our teaching and research perspectives than our individual disciplines. For most of us, almost our every word and thought about either teaching or research is grounded in our experience in the discipline. The research methods traditional in a discipline tend to influence the approaches faculty in that discipline take toward SOTL (and what journals in that discipline will accept for publication as SOTL). An interesting experiment is to pose an issue for investigation to a cross-disciplinary group of faculty members. If encouraged to talk about approaches to the investigation, their approaches might be very different, reflecting their different disciplinary homes. Often each approach has potential to shed light on the issue. The scholarship that is most effective in advancing teaching and learning may be that which combines a variety of approaches. But there is a danger. Rather than achieving synergy between different approaches, we sometimes tend to discredit or reject approaches which are not familiar to us. I felt I was moving between two groups … who had almost ceased to communicate at all, who in intellectual, moral and psychological climate had so little in common … literary intellectuals at one pole – at the other scientists …Between the two a gulf of mutual incomprehension – sometimes hostility and dislike, but most of all lack of understanding. They have a curious distorted image of each other. Their attitudes are so different that even on the level of emotion, they cant find much common ground. C. P. Snow* *Snow, C. P. (1959) The Two Cultures and the Scientific Revolution. Cambridge University Press pp.2-4

110 Quantitative and Qualitative Methods as Points of Possibility Quantitative empirical, statistical, comparative hypothesis testing, confirmatory predetermined, fixed large, representative scores, percentages, counts, rates outsider, non-perturbing deductive summative, precise, reliable Qualitative naturalistic, fieldwork, constructivist descriptive, generative, finding meaning flexible, evolving small, purposeful interviews, observations, writings insider, perturbing inductive formative, rich, expansive Feature nature goal design sample measures investigator analysis method findings As in previous comparisons, we do not go to the wall in defense of the absolute correctness or comprehensiveness of the above table entries. Nevertheless, they are offered as generally useful characterizations.

111 Typical Measures Associated with Quantitative and Qualitative Methods Qualitative performances (possibly recorded on tape) interviews (possibly recorded on tape) focus groups student projects, term papers, essay items or exams reflective statements journals reports of others (counselors, etc.) Quantitative course exam, project, paper scores survey scores (Likert) frequencies of multiple choice test item responses scores on standardized scales and tests counts (participation, web requests, office visits) measures of time use institutional research data (GPAs, grades, admissions scores, frequency distributions of demographics) In general, the more ways of gathering information and the more frequent the events in which information is gathered, the greater the validity of the study.

112 Assessment Measures Affective measures –muddiest point, mid-course survey Process measures –lecture attendance, Web hits/requests on learning activities Performance measures (for both the Class and the Instructor) –Fall 2000 compared to Fall 99, 98, 97 semesters A Neophytes Adventures in the Scholarship of Teaching Examples of Measures in a Particular SOTL Project Review the introduction to Valerie OLoughlins project Review Valeries research questions Valerie discusses content of the box below In the next 6 frames of the tutorial, some specific measures in Valeries study and the information gained from them is surveyed. Skip over the examples of measures from Valeries study.

113 Examples of Measures in a Particular SOTL Project: Mid-Semester Evaluation –Lecture notes on Web very helpful –More learning activities requested –Extra lecture review sessions requested –Review sheets requested –Wanted more exams that cover less material A Neophytes Adventures in the Scholarship of Teaching The mid-semester evaluation was a questionnaire designed by Valerie that students completed individually or in groups during class time and submitted anonymously. Key findings were: The second finding confirmed that students liked the learning activities (which were the primary intervention in the study) and wanted more of them. Valerie granted the requests in 2, 3, and 4, during the semester of her study and is considering 5 for future semesters.

114 Examples of Measures in a Particular SOTL Project: Count of Web Hits on Learning Activities A Neophytes Adventures in the Scholarship of Teaching Are students really accessing the learning activities? Lecture Requests: -Integument: 315 -Myology: 312 -Muscles of Upper Limb: 311 -Nervous System: 315 Learning Activity Requests: -Epithelium: 237 -Myology 1: 159 -Muscles of Upper Limb: 155 -Neurons: 136 Average Number of web requests per student: Fall 1998: 116 Fall 2000: 176 Web hits were counted as a measure of student use of learning activities outside of class. The left and right columns above respectively show number of requests for lecture notes and the associated learning activity during a particular month (September). The data below show that overall web hits associated with the target course increased sharply in the semester in which the learning activities were implemented.

115 Examples of Measures in a Particular SOTL Project: Pre-course data comparisons SAT-VSAT-MGPACredit Hours Grand Avg A Neophytes Adventures in the Scholarship of Teaching The purpose of background measures is to investigate whether students across semesters are fairly similar. This controls for the possibility that any improvement shown on outcome measures in the study is because the class using the intervention consisted of higher achievers than the comparison group. In this case, the higher achievers appeared to be in the 1998 class rather than the class involved in the intervention (2000).

116 Examples of Measures in a Particular SOTL Project: Common Exam Item A Neophytes Adventures in the Scholarship of Teaching Motor neurons are examples of ______ neurons. a.Bipolar b.Afferent c.Association d.Pseudounipolar e.Multipolar* A common exam item can serve as a measure of learning. Except for minor changes in distractors, this item was the same over four years of tests in the target course. In the SOTL project (2000), a learning activity about types of neurons was done in class.

117 Examples of Measures in a Particular SOTL Project: Mean Exam Performance Exam 1Exam 2Exam 3Exam A Neophytes Adventures in the Scholarship of Teaching Exam scores are frequently used as measures of learning. For a comparison of exam scores across years to have any meaning, the comparability of the exams across the years must be demonstrable. The comparability of exams was demonstrable in this study and students scored higher on 3 of 4 exams during the semester (Fall 2000) in which the learning activities were employed.

118 Examples of Measures in a Particular SOTL Project: Instructor Evaluations The format of lectures, notes and learning exercises appealed to my style of learning She made learning easier by using different teaching techniques. She gave more personal attention in a class of 250+ than many do with much smaller classes. She consistently made us feel that she wanted us to succeed and that she would go the extra distance to make that happen. Dr. OLoughlin is an excellent teacher. I speak as a humanities student who might turn to science if every science instructor were like her. A Neophytes Adventures in the Scholarship of Teaching Fall 2000 notables: -highest overall mean evaluation -highest score for question my instructor clears up points of confusion for me -highest score for question my instructor facilitates discussion among students The instructor evaluation is commonly used as both a quantitative and a qualitative measure. An end- of-course evaluation alone may not make a very compelling body of evidence in a study. However, when the evaluation is used in conjunction with a variety of quantitative and qualitative measures as in this SOTL project, a clear and compelling work of scholarship may result.

119 Guiding Questions in Choosing Methodology What approach fits your research problem? Qualitative case study Quantitative study enhanced by qualitative data Qualitative study enhanced by quantitative data A good primer: User Friendly Handbook for Mixed-Method Evaluations (NSF)User Friendly Handbook for Mixed-Method Evaluations (NSF) Why such a publication from the premier organization representing the sciences? Because of the recognition that by focusing primarily on quantitative techniques, evaluators may miss important parts of a story. Similarly by focusing exclusively on qualitative methods, researchers may miss opportunities to back their findings with the kind of objective evidence that makes the findings credible to a broader audience. Guiding questions provided by Samuel Guskin, Professor Emeritus, School of Education, Indiana University Do you have the skills/resources to carry out the methods? The skills/resources needed to employ qualitative and quantitative methods appropriately tend to be very different. Pooling of skills and resources is a good reason for community and collaboration in a campus SOTL initiative. Will your audience find these approaches acceptable? Editors or audiences within a discipline may be disinclined to accept work that does not include methods familiar to that discipline.

120 Task L: Designing Your SOTL Project 1.Individually design a study to address the question you framed in Task K.Task K 2.Discuss your tentative design with with colleagues (perhaps the same ones who helped refine your question.) Encourage your colleagues to question you and comment. 3.Sketch any revisions to your design at the end of this process. Like Task K, This task may take awhile to complete. Your design may involve sketches, or other graphics and tables or charts. Completing this task is an important step in processing the content of this unit. It is also a critical step if you plan to undertake a SOTL project!

121 Summary of Standards Clear Goals Does the scholar state the basic purpose of his or her work clearly? Does the scholar define objectives that are realistic and achievable? Does the scholar identify important questions in the field? Adequate Preparation Does the scholar show an understanding of existing scholarship in the field? Does the scholar bring the necessary skills to his or her work? Does the scholar bring together the resources necessary to move the project forward? Appropriate Methods Does the scholar use methods appropriate to the goals? Does the scholar apply effectively the methods selected? Does the scholar modify procedures in response to changing circumstances? Significant Results Does the scholar achieve the goals? Does the scholars work add consequentially to the field? Does the scholars work open additional areas for further exploration? Effective Presentation Does the scholar use a suitable style and effective organization to present his or her work? Does the scholar use appropriate forums for communicating work to its intended audiences? Does the scholar present his or her message with clarity and integrity? Reflective Critique Does the scholar critically evaluate his or her own work? Does the scholar bring an appropriate breadth of evidence to his or her critique? Does the scholar use evaluation to improve the quality of future work? Glassick, C.E., Huber, M.T., Maeroff, G.I, Scholarship Assessed: Evaluation of the Professoriate. Jossey-Bass. The six general standards for scholarship below seem an excellent wrap-up of our work in the tutorial. George Walker comments on standards for SOTL George Walker standards for SOTL

122 Task M: Closing Evaluation of Tutorial 1.My grasp of SOTL from Alpha to Omega is improved. 2.The tutorial had fresh significant perspectives of value. 3.I gained valuable resources for future use. 4.The tutorial was worth the time I spent working through it. SD D U A SA Please complete the above evaluation and press the button to responses. For example, if you strongly disagree with item 1, please reply 1SD, etc. Your comments are also most earnestly solicited. Thanks very much.

Download ppt "Almost every academician has heard the term scholarship of teaching. However, exchanges among colleagues from many institutions reveal a heterogeneity."

Similar presentations

Ads by Google