Presentation on theme: "Almost every academician has heard the term “scholarship of teaching"— Presentation transcript:
1Welcome to a Tutorial on the Scholarship of Teaching and Learning (SOTL) Almost every academician has heard the term “scholarship of teaching.” However, exchanges among colleagues from many institutions reveal a heterogeneity of meanings, attitudes, and experience levels with this topic. A comprehensive and coherent framework seems needed to accommodate the diverse strands of thought and endeavor. Similarly, a treatment of SOTL-related topics that accommodates a broad continuum of familiarity and experience seems called for.The tutorial attempts to meet those needs, by providing a “primer” … a useful grounding in the whole domain of SOTL-related topics.“The tutorial speaks to so many of the interests and concerns of faculty members and administrators who are grappling with issues of teaching, scholarly teaching, and the scholarship of teaching and learning. You and your colleagues have done a tremendous job in organizing and presenting these materials.” Barbara CambridgeA group making the journey together along the continuum “from Alpha to Omega” will find interesting topics for discussion and rich resources for further study. The tutorial is well enough indexed to also serve encyclopedically for those who seek information about specific SOTL-related topics.Materials for the tutorial originated in workshops and conferences sponsored by the American Association of Higher Education (AAHE), the Carnegie Foundation for the Advancement of Teaching, the Professional and Organizational Development Network in Higher Education (POD), the Lilly Conferences on College Teaching and various individual colleges and universities.Your hosts: Samuel B. Thompson, Craig E. Nelson, and Rita C. Naremore
2Using the TutorialClick at any point on the screen to advance to the next frame. Move your mouse to show faint arrows at the lower left corner of the frame. Click on either of these arrows to show a menu. To move backward one frame, click on (Previous).Jumps to frames elsewhere in the tutorial are dark blue and underlined. To return to the frame from which you jumped, use (Go, then Previously Viewed). This two-step procedure will probably be frequently used. Click on (Go then Slide Navigator) to see a menu of frames and to move to any point in the tutorial.To end the tutorial, use (End Show)Other types of embedded resources linked to the main path are indicated by icons:Website VideoAudioRespond to a task (click on icon)“It’s really impressive to see how much you have compressed and organized into a manageable ‘package.’ I confess I was a bit daunted by the number of frames at first, but the tutorial moves along very smoothly. A very efficient way to learn this stuff.” Pat HutchingsThis is a multimedia tutorial including photos, graphics, hyperlinks to sources, audio and video clips, and interactive tasks as well as text.The tutorial is organized in frames (not pages) consisting of short blocks of information about the topic in the frame title. The continuum is traversed by proceeding serially along a main path through the frames, stopping to examine embedded resources whenever they seem of interest. There are many frames, and it is not intended that the viewer intent on mastery of content will travel through all of them in one, or even two sittings. However, we have tried to make the journey entertaining as well as informative. Travel through the frames goes quickly, information is conveyed in various ways, and many good stopping places are evident.
4Task A1. How would you describe your experience with scholarship of teaching and learning?Choose one: Extensive Moderate Hardly-Any None2. OUTCOMES: List 3 topics or issues you hope to learn more about from this tutorialSTOP NOW! BEFORE YOU GO FURTHER, WE NEED YOUR RESPONSES TO THIS FRAME. This task is DESIGNED to capture experience level of users and their expectations/goals in freeform before influence by tutorial content. If you will be so kind as to activate the button in the slide and type your responses, we’ll put the information you provide to good use!
5Continuum of Experience with SOTL A Metaperspective ΑlphaOmega1A What & Why of SOTL1B Origins of SOTL2A Initiating SOTL Programs2B Faculty SOTL Projects3A Bridges to Productivity3B Questions, Designs & ResourcesBoyer coined the term “scholarship of teaching” more than ten years ago. The American Association of Higher Education (AAHE) and the Carnegie Foundation for the Advancement of Teaching have conducted initiatives to advance this scholarship for several years. Now, a continuum of experience levels is evident among academicians. At the “Alpha” end of the continuum, people have heard the term but are unsure what it means. These need perhaps a glossary and basic grounding in the key developments over the last decade. In the middle are people who may have launched campus initiatives or may be wondering whether particular issues and projects could result in scholarly products. At the “Omega” end, those at the “cutting edge” of SOTL endeavor to construct and validate developmental processes as sustainable bridges to carry faculty from initial interest to stages of productivity that achieve goals and count in reward systems.This tutorial addresses the whole continuum of SOTL experience from “Alpha to Omega.” The three sections of the continuum correspond to the three units of this tutorial and each phrase names a module … so two modules per unit, six total. The first unit, features essential definitions, concepts, rationale and other background. The second unit features administrative “how to” and examples of campus programs as well as faculty projects. The third unit provides some research essentials to permit viewers to launch their own projects. The “metaperspective” afforded in considering the whole continuum permits viewers to structure their own projects and campus initiatives.
6Alpha to Omega Continuum:Contents Unit 3AHow Could I Do SOTL (Genres)Task G: Reflecting on GenresApproaches to Scholarship Via Classroom Research ProjectsWhere to Publish and PresentSources of External FundingTask H: Classroom ResearchUnit 1A“The “What” of SOTLWhy SOTL?Task B: Reasons to EngageUnit 2AA Campus Example in DetailTask D: Refining Your Campus ProgramExamples on Other CampusesTask E: Applicable FeaturesUnit 1BOverviewReform ConceptsImplementing EntitiesRecent ArticulationsTask C: Critique of Originsand EvolutionUnit 2BDonetta Cothran – KinesiologyValerie O’Loughlin – Medical SciencesDavid Pace – HistoryWilliam Becker - EconomicsRita Naremore – Speech and Hearing SciencesDennis Jacobs – ChemistryLeah Savion – PhilosophyOther Examples – Carnegie ScholarsTask F: Minute PaperUnit 3BFraming QuestionsTask J: Examining Valerie’s QuestionsTask K: Framing Your QuestionDesigns for StudiesChoosing MeasuresGuiding Questions in Choosing MethodologyTask L: Designing Your ProjectSummary of Standards
7Unit 1A The What and Why of SOTL “The What”: A Conceptualization of Teaching-Related ActivitiesWhy SOTL?Task B: Your reasons to become engaged in SOTLWe seek understanding of what SOTL is and its relationship to teaching as well as reasons for becoming engaged in SOTL.
8A Conceptualization of Teaching Related Activities Many conceptualizations of SOTL have been offered over the years. This one is ours. Others will be exhibited in the next module.Every teaching-related act can be represented as a point somewhere within the three elliptical regions. Elliptical shapes are chosen solely for convenience of representation. All deeds that teachers perform in the name of carrying out their teaching assignments can be imagined as individual points within the large ellipse labeled “teaching.” The centillions of such points include acts of materials development, grading, individual tutoring … every teaching activity in or out of the classroom. Within this big red ellipse of teaching acts, we can identify a subset that meet special criteria for “scholarly teaching.” The smallest ellipse (yellow) represents “scholarship of teaching and learning.” Scholarship of teaching and learning (SOTL) is rooted in teaching but extends beyond it (out of the red ellipse). SOTL involves creation and dissemination of original work that makes a useful contribution to knowledge and practice of other teachers.More about Lee Shulman More about Pat HutchingsThough sharply defined in the picture, the boundaries of ellipses can seem a bit fuzzy when classifying certain teaching-related acts. This occasional fuzziness does not lessen the usefulness of this seemingly simplistic conceptualization.
9Relationship to Excellence Faculty members should be encouraged to move acts of teaching that are characterized by points in the red toward the orange … in other words toward scholarly teaching. To the extent that teachers adopt practices of scholarly teaching, their teaching will probably improve. Similarly, their acts of scholarship will probably improve their teaching, develop them professionally, and add to our body of useful knowledge about practice. However, we must stop short of accepting either the proposition that scholarly teaching implies excellent teaching or the converse. Similarly scholarship of teaching does not imply excellent teaching.“Excellence” is often confounded with “scholarly teaching” and “scholarship of teaching.” Scholarly teaching practices do not necessarily confer excellence. All seasoned faculty know of colleagues who are excellent teachers … excellent in terms of achieving superb outcomes in students … without their meeting any of the criteria of “scholarly teaching” Avoidable indignation and opposition to SOTL initiatives are justifiably engendered in faculty by the notion that “scholarly teaching” and “scholarship of teaching” are necessarily what one must do to be considered excellent in teaching.No ascending hierarchy of excellence can necessarily be associated with movement from left to right or bottom to top in the Venn diagram. To represent excellence, we must invoke a third dimension perpendicular to the plane of the diagram. In this three-dimensional conception, each teaching-related deed represented by a point in the diagram has associated with it an arrow (a “quality vector”) representing excellence (or lack of it). The direction of the arrow indicates whether the deed is above or below average in quality and the length of the arrow indicates how far above or below average.
10SOTL as Academic Activity vs. Campus Initiative Into what realm beyond teaching does scholarship of teaching extend? The rightmost ellipse is the set of points representing deeds that scholars perform in the name of research and creative activity. Though research-based activity to advance knowledge is the prevailing notion of scholarship, George Walker tells us that scholarship is not precisely synonymous with research. Scholarship is reflective and creative. It can result from reflecting on the processes and outcomes of research or personal teaching experience and creating something new.Boyer’s notion that scholarship of teaching and learning spans the “tired teaching vs. research debate” is naturally depicted in the position of the yellow ellipse (with italicized text) between the two big red ellipses.We can distinguish between SOTL as an academic activity and as a campus initiative. As an academic activity, what scholarship of teaching and learning means in your institution defines the interior and boundary of the yellow ellipse. As a campus initiative, what SOTL means may encompass the entire 3 leftmost ellipses. In other words, a SOTL initiative may include all teaching related activities of faculty. Thusly conceived, a SOTL initiative can provide a resource-rich and opportunity-laden home to faculty members who aspire SIMPLY to teach excellently as well as to those who perceive themselves primarily as scholars (i.e.”researchers”). In fact, there is evidence that faculty who have not been attracted to previous teaching improvement initiatives can be attracted to a SOTL initiative because of its research orientation. This phenomenon may be visualized in the diagram as faculty converging to SOTL not only from the big red ellipse on the left but also from the one on the right.
11Why SOTL?SOTL can improve teaching and learning … and more! SOTL has power to:Unite Where Teaching and Pursuit of New Understanding are Shared InterestsBuild Multidisciplinary CommunityCapture the Wisdom of PracticeFoster Mentoring of New Generations of TeachersEmploy Reserve Intellectual CapacityRecreate the Concept of Scholarly ExcellenceElaboration of the unifying potential of SOTL (Voice of Samuel Thompson)Elaboration of community-building via SOTL (Voice of Samuel Thompson)Documentation of the deep and useful insights gained over time so that others can use them is encouraged by SOTL. The documentation includes the teaching moves and materials that every seasoned faculty member has found to work especially well.SOTL is a medium for the generativity that develops naturally in mid-life … seasoned teachers helping newer ones to learn to teach more effectively.SOTL may increase the individual and collective scholarly productivity of the faculty, partly by providing an avenue to products from the immense investments that already go into teaching and partly by providing an area where faculty productivity can thrive when scholarly efforts in other areas are flagging or no longer interesting.SOTL brings the spirit and standards of scholarly excellence to teaching .. standards that have traditionally been applied to other fields of scholarly endeavor but not to teaching.
12Why SOTL? (Cont.)SOTL Honors and Enriches the Seasons of Academic Life Elaboration of the role of SOTL in the seasons of academic life (Voice of Craig Nelson)Power of IdeasFaculty RoleStudentsPublic SelfMultiple and Competing CommitmentsNeed to Step OutMarginalityCourageKnefelkamp L. L. (1990) Seasons of Academic Life: Honoring Our Collective Autobiography Liberal Education 76. 3Knefelkamp’s characterization of Seasons of Academic Life seems worthwhile reading for any faculty member. This particular article may serve as a basis for a meaningful exercise in deepening understanding of “Why SOTL?” The recommended exercise is to read Knefelkamp’s description of each season and then pause to reflect on the possible roles that scholarship of teaching and learning may take during it and the implications for the faculty member.
13Task B: Reasons to Engage in SOTL What reasons to become engaged in SOTL … the ones discussed in the tutorial or others … are most important on your campus?What reasons might be irrelevant to faculty on your campus?We pause here for processing of the foregoing and for reflection. Please activate the button and your reasons. We’ll compile categories of reasons and disseminate the results.
14Unit 1B Origins and Evolution of SOTL OverviewReform ConceptsImplementing EntitiesRecent ArticulationsTask C: Critique of Origins and EvolutionAs we progress to the second and subsequent modules, we’ll continue to broaden and deepen our concept of “what SOTL is” and “why become engaged in SOTL.”We invite your peer review and critique of this module. At the end of the module, we will ask you (Task C) to tell us how you would modify the framework and which elements you would add or omit in each the first four categories above. We’ll consider your inputs and may incorporate them in future versions of this tutorial in order to build a better framework for everyone.
15Origin and Evolution of SOTL (In Progress) Not a new idea (Hutchins 1923)“Scholarship of teaching” coined (Boyer 1990)Implementing EntitiesAAHECarnegiePewLillyReform ConceptsClassroom Assessment/ ResearchNew EpistemologyNew American ScholarConception of TeachingScholarship AssessedRecent ArticulationsScholarship of TeachingScholarly TeachingModel of SOTLRelation of ST to SOTLThe “work in progress” in this “tree” is our tentative attempt to build a succinct framework for essential background in SOTL. Of course, the number of individuals, forums, articles, and books that have significantly contributed to SOTL is too enormous to include comprehensively. And who is to say which of these is “essential”? So this whole exercise is somewhat arbitrary and no doubt reflects the bias of our own limited knowledge and experience.
16Not a New Idea“What is certain is that most Ph.D.’s become teachers and not productive scholars as well. [A Ph.D. candidate who plans to be a teacher] must know his field and its relation to the whole body of knowledge. It means too that he must be in touch with the most recent and most successful movements in undergraduate education, of which he now learns officially little or nothing. How should he learn about them? Not in my opinion by doing practice teaching upon the helpless undergraduate. Rather he should learn about them through seeing experiments carried on in undergraduate work by the members of the department in which he is studying for the degree….”Robert Maynard HutchinsThe quote is taken from Hutchin’s inaugural address as the fifth president of the University of Chicago in 1928 and is presented as evidence that the concept of SOTL existed long before Boyer popularized the term. The last sentence suggests that faculty in all departments should carry on experiments in undergraduate teaching and learning … a fundamental activity of SOTL. The sentence further suggests that graduate students preparing for academic careers should learn from these experiments.Note that Hutchins also presaged the plight of many undergraduates, and the need for programs to prepare graduate students to teach … programs like the Preparing Future Faculty (PFF) initiative of the 1990’s.In the first sentence, Hutchins also expressed a truth still not internalized by many faculty who prepare graduate students in doctoral institutions today … research productivity that advances the discipline ends with the dissertation for most Ph.D’s. Today, the vast majority (95% is a figure often quoted) of new Ph.D’s who take positions in academe do so in teaching institutions rather than research universities. In other words, though they may acquire their Ph.D’s in research universities as researchers, teaching is what they will primarily do in academic careers.
17“Scholarship of Teaching” Coined Ernest Boyer popularized “scholarship of teaching” in a famous monograph Scholarship Reconsidered: Priorities of the Professoriate Princeton, NJ: The Carnegie Foundation for the Advancement of Teaching.Boyer argued that the concept of scholarship must be broadened to include not only basic research but other kinds of intellectual work in which faculty engage. He suggested four types of scholarship: discovery, integration, application, and teaching. All of the elements of “Reform Concepts” build on Boyer’s theme of broadening the traditional view of scholarship.Boyer disseminated the notion of a scholarship of teaching but did not define clearly what this scholarship would be. He thereby touched off a decade of academic thought and controversy over this topic. As a result, our conceptualization of this scholarship is a bit clearer today (but only a bit!)
18Reform Concepts: Classroom Assessment/ Classroom Research The several reform concepts presented in this section of the tutorial may be viewed as multiple origins or histories of SOTL. They are somewhat overlapping and sometimes difficult to isolate.Classroom Assessment Techniques (CATs)K. Patricia Cross and Thomas Angelo pioneered this “do-it-yourself” approach to assessing conditions of learning in classrooms. Their 1993 book, Classroom Assessment Techniques: A Handbook for College Teachers (2nd Ed) Jossey-Bass is a famous work of scholarship of teaching and learning, reported to have sold more than 35,000 copies. Thousands of college and university teachers use CATs in their classrooms today, especially versions of the “one-minute paper,” and there is evidence of improved learning as a result. Angelo and Cross contributed to the current focus on learning but also hoped their book would influence faculty to go beyond application of CATs in the classroom … that it would in fact spark systematic classroom research which would in turn become a popular form of SOTL nationally … a form more useful to faculty than traditional educational research. The adoption of CATs became widespread, the adoption of classroom research much less so.Classroom Research In 1996, Cross with Mimi Steadman published a sequel book, Classroom Research: Implementing the Scholarship of Teaching, Jossey-Bass placing CATs in a broader research context. Again the hope was to institutionalize classroom research. However, as of yet, classroom research is still not nearly as widely employed as are CATs.More on CATS More on Classroom Research
19Reform Concepts: New Epistemology “On the high ground, manageable problems lend themselves to solution through the use of research-based theory and technique. In the swampy lowlands, problems are messy and confusing and incapable of technical solution. ...the problems of the high ground tend to be relatively unimportant to individuals or to the society at large, however great their technical interest may be, while in the swamp lie the greatest problems of human concern. ...Shall [the practitioner] remain on the high ground where he can solve relatively unimportant problems according to his standards of rigor, or shall he descend to the swamp of important problems where he cannot be rigorous in any way he knows how to describe?”Donald SchonDonald Schon posed a “dilemma of rigor or relevance” for research epistemology. In this eloquent quote, he calls for new research epistemology for the scholarship of teaching and learning based on reflection by teachers on actions taken in their practice.Schon, D. “The New Scholarship Requires a New Epistemology,” Change, Nov./Dec ( 6) p. 28
20Reform Concepts: The New American Scholar Adapting the learning cycle of David Kolb to SOTL, Gene Rice sees traditional scholarship … particularly the so-called “social science model” … as characterized by reflective observation and abstract analytic knowing. It therefore resides in the lower right quadrant of his diagram. He asserts that SOTL needs to be rooted more in active practice and concrete connected knowing. Said another way, scholarship of teaching and learning should take forms that properly reside in the upper left quadrant
21Reform Concepts: Conception of Teaching & Scholarship Assessed Conception of Teaching: The argument here … and Lee Shulman is a major proponent … .is that teaching is not just technique (though technique has gotten the lion’s share of attention in teaching-improvement efforts) but an enactment, rather, of our understanding of our disciplinary, interdisciplinary or professional field and what it means to know it deeply. As Shulman wrote recently "a scholarship of teaching will entail a public account of some or all of the full act of teaching—vision, design, enactment, outcomes, and analysis—in a manner susceptible to critical review by the teacher’s professional peers, and amenable to productive employment in future work by members of that same community."Scholarship Assessed: Glassick, Huber, and Maeroff (1997) in Scholarship Assessed: Evaluation of the Professoriate. Jossey-Bass review the history of scholarship in American academe and describe general standards for all forms of scholarship. These standards “pull together” many of the concepts noted in this section. More detail on the standardsSummary of Reform Concepts. All reform concepts contain an implicit call to broaden and generalize traditional notions of scholarship in order to make progress against relevant problems which vex undergraduate teaching and learning at levels of individual teacher and course. George Walker advises us to avoid unnecessary limitations in our notions of scholarship. The goal is to produce a body of scholarship that may deviate from conventional forms, and that may not be applicable over disciplines, categories of student, or institutional type but which nevertheless is more used by teachers and contributes more to the learning of students than does past educational research.
22Implementing Entities: AAHE Implementing Entities We focus on broad national initiatives. There are far more entities (including many academic institutions) deserving acknowledgement for implementation than those included here.AAHE (American Association of Higher Education) The AAHE has sponsored many initiatives over the years to advance teaching and its scholarship. Barbara Cambridge directs the Teaching Initiatives. Major initiatives include assessment, service-learning, peer-review of teaching and teaching-learning technologies. We highlight faculty “roles and rewards” below because its historic origin is associated with Boyer’s introduction of “scholarship of teaching.”AAHE FFRR (Faculty Forum Roles and Rewards) It was recognized at the time of Boyer’s 1990 monograph, that overhaul of the faculty reward system needed to go hand-in-hand with advancement of scholarship of teaching, if the latter was to be more than a transient topic. The FFRR has met annually since 1990 to address faculty roles and rewards.Robert Diamond’s depiction of the factors bearing on faculty work reflects the agenda of AAHE FFRR. Initiatives have been undertaken in the 1990s’ to favorably influence each of the five factors in the figure toward greater incorporation of SOTL in faculty work.Factors Influencing What Faculty DoDiamond, R.& Adams, B. The Disciplines Speak AAHE 1995 p.7
23Implementing Entities: Carnegie THE CARNEGIE FOUNDATION FOR THE ADVANCEMENT OF TEACHING The Carnegie Foundation has a number of exemplary programs and resources that encourage scholarly teaching and the scholarship of teaching and learning. Among the programs that may be explored are: Cultures of Teaching and Learning in Higher Education, Higher Education and the Development of Moral and Civic Responsibility (co-led by Tom Ehrlich, former President of Indiana University), Preparation for the Professions, Teacher Education, and the long running U.S. Professors of the Year. Four key programs to click on:Rethinking the Doctorate. George E. Walker, Vice President for Research and Dean of the University Graduate School at Indiana University, leads this new, five-year project.Knowledge Media Laboratory. A gallery of course portfolios illustrating various approaches to the scholarship of teaching. Relation between course portfolios and SOTLResources. Click on “e-Library” to read key SOTL references on-line and to access an annotated bibliography of SOTL in higher education.CASTL Carnegie Academy for the Scholarship of Teaching and Learning
24Implementing Entities: Carnegie (cont.) CASTL Carnegie Academy for the Scholarship of Teaching and Learning (CASTL)Pew National Fellowship Program for Carnegie ScholarsFor faculty from higher education - brings together outstanding teacher-scholars from a variety of academic fields and institutions to pursue investigations of issues in teaching and learning.Teaching Academy Campus ProgramCoordinated for the Carnegie Foundation by AAHE. Institutions in higher education that are prepared to make a public commitment to the scholarship of teaching and learning are invited to join the Campus Conversations program. As progress is made, campuses are invited to apply for small grants for the Going Public phase. CASTL now is also encouraging the formation of National Networks of Teaching Academy Campuses. The AAHE CASTL Campus Program Web Center includes a number of resources for thinking about SOTL, including reports from campus conversations, a list of going public grants and selected resources.Work with the Scholarly and Professional SocietiesCarnegie is encouraging societies to increase their emphasis on teaching and on the scholarship of teaching and learning. This is a continuation of work under AAHE FFRR culminating in two publications which contain perspectives of disciplinary societies on Boyer’s four scholarships: The Disciplines Speak AAHE, 1994 and The Disciplines Speak II AAHE, 2001.
25Implementing Entities: Pew Peer Review of Teaching Course Portfolios Pew Charitable Trusts:CASTL (with Carnegie Foundation)Peer Review of TeachingPeer Review of Teaching is an ongoing area of work intertwined with the scholarship of teaching and learning in several ways.Peer Review has been an emphasis of AAHE since and continues to be supported by the Pew Foundation and other grant sources. The course portfolio has evolved as the principal means by which peer review of teaching is effected. Two notable current programs to advance course portfolios, both supported by Pew, are centered at the University of Nebraska Lincoln and Samford UniversityIndividual course portfolio projects and SOTL projects are so closely related as to often seem inseparable. Example of this inseparability.Perhaps the best way to ensure that quality teaching is recognized, valued, and rewarded is to improve the means of identifying and documenting teaching effectiveness. Course portfolios afford a comprehensive yet efficient means of documenting the intellectual work of teaching a particular course. Through such a portfolio, a faculty member documents course design and execution, including results in student learning. In this way, teaching can be understood and presented as a form of scholarship, utilizing the accountability through peer review that already exists in higher education. A course portfolio can be used as an instrument for exhibiting teaching effectiveness, a framework for cultivating scholarship, and a vessel for conveying one's work to appropriate publics, including promotion and tenure committees.
26Implementing Entities: Lilly Conferences on College Teaching Lilly Conferences have encouraged scholarly work in teaching and provided a venue for faculty members to discuss and disseminate such work for more than 20 years.Lilly Conferences are retreats that combine workshops, discussion sessions, and major addresses, with lots of opportunities for informal discussion about excellence in college and university teaching and learning. Internationally-known scholars join new and experienced faculty members and administrators from all over the world to discuss topics such as gender differences in learning, incorporating technology into teaching, encouraging critical thinking, using teaching and student portfolios, implementing group learning, and evaluating teaching. The welcoming "Lilly Spirit" and the high level of scholarly attention to teaching and learning enable everyone to contribute to the forum.
27Recent Articulations: Scholarship of Teaching Scholarly Teaching Scholarship of Teaching and LearningThis recent articulation (Shulman,Change 31 (4) 1999) is widely accepted. Note that it is not quite a definition; it offers necessary conditions for scholarship but not sufficient conditions. The articulation does suggest a broadening of the concept of scholarship as advocated in the Reform Concepts Applications of the broadened concept of scholarship.Scholarly TeachingThis articulation (Hutchings & Shulman Change 31 (5) 1999) is also widely accepted. One of it’s principal features is to include knowledge of recent and relevant developments in pedagogy as well as the traditional notion of recent and relevant developments in the field as a criterion for scholarly teaching.We have left the topic of “Implementing Entities”, and moved to “Recent Articulations” (the rightmost branch of the “origin and evolution tree”. ) A decade after Boyer’s “Scholarship Reconsidered”, the meaning and implications of scholarship of teaching and learning are topics of evolving thought and discussion. Many more entries than the four shown could be included in the right branch and these would be no less worthy than the sources shown. However, our purpose is to provide a sample rather than a comprehensive list. Our apologies to authors for failing to list their contributions.
28Recent Articulations: Exploring Scholarship of Teaching The Model of Kreber & Cranton Carolin Kreber and Patricia Cranton see development of scholarship of teaching as a process of reflection on experience and research in teaching.The second, third and fourth concentric circles represent content, process, and premise reflection respectively. The three spokes partition the circles into regions representing the three domains of knowledge (instructional, pedagogical, and curricular) shown on the periphery of the figure.Within each knowledge domain, learning is characterized by various forms of instrumental, communicative, and emancipatory learning processes, which are derived from Habermas’s three forms of knowledge.Development of SOTL in this conceptualization is thus comprised of nine distinct types of learning.
29Recent Articulations: Conceptualizing Scholarly Teaching and Scholarship of Teaching Laurie Richlin’s conceptualization reflects her extensive experience as co-director of the Lilly Conferences on College Teaching and in helping faculty with teaching-related investigations. Her conceptualization shown in the slide is closely tied to the ideas of classroom research. She partitions a developmental process for SOTL according to the aspects that can be considered scholarly teaching and scholarship of teaching. Her steps in the part of the process called scholarly teaching are similar to those in our classroom research schematic.Richlin, L in Kreber, C. (Ed.) New Directions in Teaching and Learning (In Press)
30Task CThink about your own knowledge of SOTL. What additions would you make to the framework “Origins and Evolution of SOTL” in each category?Reform ConceptsImplementing EntitiesRecent ArticulationsWe pause to process, reflect on, and improve the “Origins and Evolution of SOTL.” WE NEED YOUR HELP WITH THIS! Please suggest alterations to the structural framework of the tree as well as to the specific elements in the branches. Just click on the button on the frame to your suggestions. We’ll be most grateful for your contributions, and they will result in a better product for everyone.
31Unit 2A Initiating SOTL Programs A Campus Example in Detail ( Indiana University Bloomington)Task D: Refining Your Campus ProgramExamples on Other CampusesTask E: Applicable FeaturesWe are leaving the left region (Unit 1) of the Alpha to Omega Continuum and transitioning to the middle region (Unit 2).
32A Campus Example in Detail: Using SOTL to Make Change Happen Identifying Key ResourcesGetting People InvolvedChanging the Institutional CultureAssessing SOTL Program ImpactTask D Refining Your Campus ProgramAn alternative path through this unit is to view “Assessing SOTL Program Impact” first. After studying “A Campus Example in Detail”, you will be asked to reflect on whether any aspects of this example are applicable to your own campus (Task D).
33Three Stages of Campus Program Goals ParticipationScaffoldingScholarly ProductivityCampus program development may be viewed in a framework of three stages. The first stage is to build awareness and participation. The second is to erect essential scaffolding to support faculty in this line of work. The third is to maximize scholarly productivity.It may be helpful to bear the above three stages in mind while studying this detailed example of a campus program. To assess the extent to which goals of the first two stages are achieved, look for evidence distributed through the frames of this unit. Evidence of scholarly productivity will be largely found in the following unit “Faculty SOTL Projects.”
34What are the Key Resources? The Ingredients:Massive administrative supportA dedicated directorA faculty advisory councilA core of faculty members who are willing to engage in this work and share it with othersMassive administrative supportMoya Andrews is committed to the campus SOTL initiative and an architect of it from its inception. Her administrative support is massive and visible to all. She provides a budget for the SOTL initiative. However, she also contributes vitally by staying abreast of SOTL-related developments and directing the campus initiative. She makes public statements, writes messages to faculty and invitations to department heads and other key individuals. She often introduces featured presenters at events.Message to faculty regarding SOTLMoya AndrewsDean of the Faculties and Vice Chancellor for Academic Affairs
35Massive Administrative Support (Cont.) “Expanding our Vision of Indiana University’s Research Mission: The Scholarship of Teaching and Learning”Vice President Walker’s talk at the first SOTL main event was subsequently delivered to representatives of research universities at invitation of AAHE and distributed nationally.Other SOTL resources by George Walker:A statement that SOTL is important.A brief message to faculty regarding the rationale and significance of SOTL.The campus SOTL initiative received a huge boost in administrative support when George Walker became a partner in it. From that time, the SOTL initiative was impelled by the two primary academic offices on the campus. This partnership facilitated the movement of faculty interest toward SOTL from research as well as teaching. George Walker describes some of the administrative support available for SOTL.George Walker is currently a senior scholar at the Carnegie Foundation leading a new study: Rethinking the DoctorateGeorge Walker’s contributions exemplify what a distinguished researcher and senior administrator can do to help advance teaching and its scholarship.George Walker Vice President for Research and Dean of the Graduate School
36Director of the Campus SOTL Initiative A Dedicated DirectorA campus SOTL initiative needs a person charged with responsibility to take actions on ideas, to follow up actions to ensure completion, to convene groups, to plan and stage events, to coordinate faculty and administrative viewpoints, to “cheerlead,” and to serve as a stable point of contact. If the director is a faculty member, release time is essential. The SOTL initiative needs to be a significant part of the director’s portfolio.Samuel ThompsonDirector of the Campus SOTL Initiativeto Samuel
37The Faculty Advisory Council Insure Representation Across DisciplinesTap Respected FacultyLet FACULTY Define InitiativesSupport FACULTY InitiativesSuccess may depend on the initiative being of, by, and for the faculty. Advisory council members can steer the enterprise. They can be inspired by the prospects of what the initiative can do and so contribute their own time and effort without acknowledgement in tasks which they, themselves, often define. This is “bottom up” infusion of energy.Advisory council members should be chosen carefully. Ensure membership is representative. Primarily use senior, respected faculty: they know the ins and outs of the institution and their presence validates and lends prestige to the SOTL effort.Make sure the initial SOTL thrusts on the campus reflect consensus of faculty on the advisory council. Let faculty members feel empowered to influence key decisions. Advisory council members suggested doubling the amount of the SOTL grants proposed by administration, defined and addressed the human subjects issue for SOTL, and took the initiative to bring the research establishment into partnership.Administration supported the advisory council initiatives and decisions.
38A Core of Faculty Members Willing To Engage in This Work and Share It With Others Because this unit of the tutorial deals with launching campus SOTL initiatives, much emphasis is on administrative considerations. However, all the administrative brilliance in the world will not attract or sustain faculty in a SOTL initiative if they do not see relevance to academic issues important to them.If a campus already has scholars of teaching and learning, these may provide examples or kernels of issues which engage colleagues. But what if such scholars and issues are not readily identifiable?Engaging faculty should not be a struggle. The vast majority of faculty have a natural curiosity about teaching issues and a latent desire to do teaching-related inquiry. Making time and resources for reflection and inquiry available to faculty in light of their teaching loads and other responsibilities might be a challenge but as Shulman has suggested, students already provide faculty with more information than they normally use. Thus additional work, if any, in information-gathering should be minimal.A possible two-step procedure is to conduct a kind of “needs assessment” … formal or informal … to identify campus issues for possible investigation; e.g. academic incivility. The second step is to connect an identified need to language and issues that people care about … framing the issue to make it attractive in the particular setting.If engaging faculty in issues and framing them is of concern to you, you might keep this concern in mind when viewing other examples of campus programs , the issues investigated in Faculty SOTL Projects, and Framing the Question.
39Getting People Involved Get Out the Crowd to Initial EventsShowcase Campus ScholarsIdentify Campus-wide Needs and InterestsPartner with StakeholdersKick off the SOTL initiative with a “Celebration of Teaching.” Send personal invitations to winners of teaching awards, recipients of teaching-related grants, teachers of pedagogy courses for graduate students, and all others who have demonstrated interest in teaching in an identifiable way. Send personal invitations also to deans, chairs, and P&T committee members. About 200 faculty showed up for our Celebration, more than in anyone’s memory for a teaching-related event. Personal attention has continually proved important in building a SOTL community “one faculty member at a time.”Identify and encourage faculty who wish to pursue scholarly inquiries into teaching or have already done so. Make these visible as models and mentors in high profile events rather than bringing in outside speakers.Use information-gathering opportunities to learn the needs, goals and activities of people and units across the campus. How can SOTL help to fill their needs? Focus on issues for investigation that cut across disciplinary boundaries.Seek to make the SOTL initiative inclusive of all academic and non-academic units on the campus. Seek to involve undergraduate and graduate students, professional staff in academic affairs and student affairs, librarians, and faculty in all seasons of academic life. Seek as well to support and cooperate with all other teaching-related initiatives rather than to compete with them. Think of ways and themes for collaboration that advance the goals of stakeholders as well as those of the SOTL initiative.
40Getting People Involved: An Example As an example of partnering with stakeholders, consider librarians. They generally see themselves as academics but do not always feel that others see them in that light. When invited to join and help grow the SOTL initiative, they were very pleased for the opportunity to participate with faculty in an obviously academic enterprise. The librarians enriched SOTL with talent, initiatives, and resources.Librarians:volunteered as consultants to individual faculty investigators needing assistance in locating unfamiliar teaching and learning literature.gave presentations to show use of information technology to access SOTL-related resources.made free document delivery systematically available to scholars of teaching.acquired multiple copies of critical books and put them on reserve for “SOTLites”.served on working committees of the Faculty Advisory Council.The librarians, in turn, found participation rewarding both individually and collectively.
41Getting People Involved (continued) Connect with National InitiativesThis campus model as well as all six campus SOTL models described later in this tutorial draw direction and resources from AAHE and Carnegie national initiatives … and frequently publicize the national connections to the local community. Other connections to related national initiatives include Peer Review, Problem-Based Learning, and Preparing Future Faculty.Exhibit Administrative SupportIt’s important not only to have administrative support but to exhibit it. One way is by regular participation of key administrators in SOTL events. Serving food exhibits institutional backing with funds as well as increasing participation, especially for events held at lunchtime or the end of the working day. Examples of SOTL eventsProvide Scaffolding for FacultyIf a SOTL initiative is to achieve it’s objectives, getting faculty interested or involved is not enough. Many need “scaffolding” … various forms of support … on the pilgrimage from initial interest to completion of scholarly work. Planning for this scaffolding and building it into the campus model is essential.
42Changing the Institutional Culture Change the faculty member’s view of what faculty members doUnder the aegis of SOTL, employ administrative resources to acknowledge and reward a fuller range of faculty work.Expand the range of activities that get rewardedMany campuses have faculty who have done scholarship of teaching for years. However, nobody necessarily knows who they are if such scholarship does not count in reward systems. In this campus example, small grants ($1000) were offered at the start of the SOTL initiative for scholars of teaching to present work already completed or well underway. These unusual “presentation” grants for work that may have already been done made existing scholars and their projects visible campus-wide as models. Later conventional “research” grants to undertake new investigative projects were offered.Many campuses already have small grant programs. These can be tweaked to foster SOTL. A spin-off benefit can be greater productivity for a given institutional investment in grants; e.g. a grant that was previously to implement a teaching innovation will, as a SOTL project, also require data-gathering and analysis to assess the effect of the innovation as well as dissemination of results to peers.Expand the range of rewardsOpportunities to present their work to large interdisciplinary audiences in high profile campus-wide events; to have their abstracts, biographical sketches, and photographs widely disseminated; and to experience the support and confidence of the central administration proved enough reward to some faculty members for them to offer themselves as SOTL presenters(without grants). For some faculty members, finding a community that valued their work to improve teaching and that helped them find opportunities for professional growth and recognition beyond the campus and beyond opportunities available in their departmental and disciplinary cultures also proved rewarding..
43Changing Institutional Culture: One Example IU FACULTY SUMMARY REPORT PRIOR TO ACADEMIC YEARTEACHING ACTIVITIESCourses taught (weekly contact hours reported by course number in tabular form).Development or major revision of course(s) during the year.Dissertation, Research and Field Work CommitteesTeaching awards and honors, including those of your students.Indiana University’s summary report form for individual teaching activities required of each faculty member annually was as shown above prior to Fall It is similar to that used in many institutions. There are also sections (not shown) for Research & Creative Activities and Service.Look at the categories of reportable teaching activities. Category B is generally taken to mean curriculum development or major revision of curriculum. Activities associated with improvement of pedagogy or with demonstrable evidence of improved student learning are not encouraged on such a report. Indeed, the lack of a place to report pedagogical activities on the form is a tacit statement about institutional values and rewards.
44Changing Institutional Culture: One Example (cont.) IU FACULTY SUMMARY REPORT (REVISED) IN ACADEMIC YEARAND THEREAFTERTEACHING ACTIVITIESCourses taught (weekly contact hours reported by course number in tabular form).Activities directed at improving instruction, learning, or course administration. (Please describe rationale for/description of innovations, methods/measures for assessing outcomes, and results.)Please note: Scholarly activity related to teaching and learning (e.g. investigation/research, dissemination/publication of results) should be reported under the section on Research/Creative Activities.In Academic Year , the Faculty Summary Report form for the Bloomington campus was revised to include the new Category B shown in red. The former categories B, C, and D respectively become C, D, and E on the new form. Thisrevision was prompted by the SOTL initiative.
45Changing Institutional Culture: Another Example An initiative of Craig Nelson, a faculty member committed to the campus SOTL initiative and a Carnegie Scholar, led to an institutional policy of partially funding travel of faculty members to attend teaching-related conferences. Candidates for support must show demonstrable connection to their scholarly interests in teaching and learning and provide a quid pro quo to the campus in return.Craig Nelson is a co-founder of the campus SOTL initiative. His contributions are many. It’s very helpful to have a teacher-scholar of his prominence involved!Craig Nelson Biologyto Craig
46Changing Institutional Culture Further Examples Input from the Faculty Advisory Council and interested faculty researchers influenced the human subjects process to facilitate SOTL activities.Input from the Faculty Advisory Council and interested faculty researchers facilitated access to institutional data (registrar, etc.) for SOTL researchers.These achievements were important steps in “developmental scaffolding” for faculty embarking on SOTL projects. An aversive institutional review board decision regarding students as human subjects or inability to gain access to essential background data regarding the students in the study can be lethal to a SOTL project.
47Assessing SOTL Program Impact: Participation Statistics 1999-2001 Campus Participation in SOTL Program of EventsDavid Pace provides an overview of SOTL program impact.14 main events in program; 11 in – about one every two weeks in fall and spring terms108 campus units represented ( 74 academic departments/ programs)*Participants:496 individual faculty members286 tenure/tenure track (22% participation rate for T/TT)51 individual staff 237 graduate students Total: 784 individuals (63% faculty, 30% graduate students, 7% staff)*Total attendance: 1598 (53/event in 1st year and 77/event in 2nd)**Greatly exceeding expectations of many observers
48Selected Faculty Comments about the Campus SOTL Initiative We need SOTLThe events I attended were terrific. Keep it up. And keep me in the loop.I was not able to participate directly in most SOTL events, due to numerous personal and professional obligations. But I was impressed by the degree to which SOTL spurred conversation on teaching among my colleagues. I also met with one SOTL participant to consult on his research project, and I was impressed with the enthusiasm he was bringing to his teaching and the related project. SOTL has been great for this campus (and, more importantly, our students), and I plan to participate enthusiastically next year. Thanks!SOTL is an outstanding source of intellectual support for the graduate school's efforts in "preparing future faculty."I realize that the program is in some sense intended to support and provide a forum for people producing the scholarship of teaching and learning. However, please do keep in mind how useful it can be to those of us simply working on teaching.I think you are doing a wonderful job, and I fully support your workSOTL is a very valuable asset to the university and the School of HPER. Please continue with your excellent efforts. I think you are beginning to make substantive improvements to the learning environment and are slowly raising awareness in the general faculty about the importance of teaching and how this may be approached through scholarship.I am afraid I have a rather dim view of "scholarship" about teaching or learning. Teachers should teach, learners should learn. Scholarship should be reserved for inquiry in genuine subjects.Scarce resources would be better devoted directly to teaching (e.g. smaller class sizes) or traditional substantive areas of researchThe ratio of positive to negative comments is actually much higher than in the sample here.
49Assessing SOTL Program Impact: Scaffolding Support for SOTL by research establishment provided on same basis as research in the disciplines (matching funds, summer fellowships, etc.)Campus colleagues with specialized expertise voluntarily contribute time and effort to consult with scholars of teaching.Growing constellation of linked efforts affording opportunities and support (course portfolios, PFF, etc.)Large informal “SOTL community” on campus Samuel explains why multidisciplinary community is important to SOTL (beginning at 2 minutes: 10 seconds into the audio clip)Scholarship given birth under SOTL initiative begins to gain recognition (and invitations) beyond campusA visiting scholar of teaching and learning, Professor E. F. Redish, Department of Physics, University of Maryland, remarked that he had not previously experienced such a multidisciplinary community of scholars investigating issues of teaching and learning.After meeting members of this community, Lee Shulman, President of the Carnegie Foundation for the Advancement of Teaching, echoed similar sentiments ... that this SOTL community has a truly extraordinary robustness and interdisciplinary character.
50TASK D: Refining Your Own Campus SOTL Program or Plan Think for a minute about your own campus. What administrative priorities or structures might help in your campus level SOTL program? What elements of the preceding example, if any, may be applicable? What hurdles might need to be overcome on your campus?Pause here to reflect on administrative features and obstacles on your own campus in the context of the Bloomington example. Press the button and share your thoughts. Your comments may prove useful in unforeseen ways. Besides, we’re REALLY interested!
51Other Campus Examples Elon College The Citadel Rockhurst University Abilene Christian UniversityNotre Dame UniversityMiddlesex Community CollegeTask E – Applicable FeaturesCampus examples of SOTL initiatives from six different institutions of widely varying types are exhibited in the following frames. All of these campuses are participants in the AAHE Campus ProgramAfter surveying these examples, you’ll be asked to identify features of them that might be applicable to your campus program (Task E)
52Elon College Multidisciplinary, Multiyear, $72,000 Investment $6000 projects in each of 3 yearsProjects directed by faculty-student research teamsLearning for BOTH student and teacherEight projects selected in years 1 and 2Create intellectual engagementNew thinking in diverse fieldsApplication of learning to lifeOpening spaces for reflective integrationThe magnitude of the Elon investment in the SOTL projects is notable for a school with about 4000 students. The projects focus on conditions of learning and the individual course is the unit of study. The use of student-teacher research teams to carry out each project is a distinctive feature.Elon jointly with Western Washington University won the first AAHE “Going Public” grant. John G. Sullivan, Powell Professor of Philosophy, kindly provided the information used in this tutorial. For more information
53The Citadel Highlights Effectiveness Administrative Support Mission Increased campus awareness of and participation in SOTLFocusCommunication, Resources and Continuing EducationSelf-selected research projectsHighlightsBiweekly, participatory meetings with assignmentsEffectiveness15% of full-time, tenure-track faculty at bi-weekly meetings12% of full-time, tenure track faculty in classroom researchAdministrative SupportAttendance at functionsFinancial supportThe Citadel SOTL initiative got a big boost from a Mimi Steadman workshop on classroom research. The Citadel SOTL website is rich in resources and ideas. It’s well worth a browse. One distinctive feature is the schedule of “assignments” for the year given to participating faculty.Professors Alix Darden and Suzanne Mabrouk spearhead work in this initiative. Suzanne can be ed through the website. Alix was selected as a Carnegie Scholar for toAlix
54Rockhurst University Beginnings The Rockhurst “Carnegie Seminar” (Fall 1998) All University Symposium(Spring 1999) Follow-up SymposiumYear-long Carnegie faculty seminar( ) Carnegie faculty seminar continuedThe Rockhurst “Carnegie Seminar”Central QuestionsSeminar MembersDiscussionsMethodsProductsFormal Letter on SOTLFaculty SOTL ProjectsSelected Key Issues and ObservationsObstacles to DiscussionInterdisciplinary/Collaborative Approaches“Where's the beef?”“Scholarly Teaching” as best first pathTo be a good consumer of the SOTLRockhurst, a small (3000 students) independent university in Kansas City, conducted a year-long Carnegie seminar which resulted in a formal letter addressing reward for scholarly teaching and scholarship of teaching in tenure and promotion.An observation of the Rockhurst Carnegie Seminar … which is entirely consistent with observations at other institutions … is that a focus on scholarly teaching is a good first step to scholarship of teaching.Several faculty research projects are underway at Rockhurst. One researcher is Professor Anita Salem (mathematics), also a Carnegie Scholar of Teaching. Anita provided the information for this tutorial. to Anita
55Abilene Christian University 19 Faculty Engaged in SOTL ProjectsStrong Institutional SupportStipends for Materials & ResourcesTravel to Teaching-Related Conferences-Ongoing Peer Meetings,VideoconferencesAbilene Christian provides an example of an effective SOTL initiative at very modest cost in a private institution with about 4500 students. Faculty participants make a year-long commitment to get their projects to the stage of publication. They are given small stipends for materials and resources. An outside consultant provides a day long workshop to help shape the projects. Ongoing support is provided through peer meetings and videoconferences with the consultant.Visit the ACU SOTL website or request a SOTL program brochure from Nancy Shankle, English Department Chair. Nancy steers the SOTL initiative at ACU. to Nancy
56University of Notre Dame Initial campus conversations with 90 campus leadersSOTL needed supportRFP resulted in 9 funded SOTL projectsSample research question: Do new teaching methods in introductory engineering affect students’ learning?Support for SOTL teams includes these elements:$5,000 per team for student time, equipment, supplies, faculty timeConsulting with methodology expertsGroup meetings 2x/semester for mutual supportHelp in dissemination of resultsThe Notre Dame SOTL initiative features faculty research projects, campus conversations, and a Carnegie Scholar. Barbara Walvoord and her team of “experts” provide support to investigators in regional institutions as well as at Notre Dame to BarbaraNotre Dame’s approach – to develop a community of scholars and define SOTL by example – is similar to the approach in Bloomington. The impediments are similar too … like faculty reluctance to engage in a scholarship for which they feel they have no training.
57Middlesex Community College Interdisciplinary group of 9-10 facultyReceive 1 course release for minimum 2-year commitmentMeet biweekly for two-hour seminarsStudy SOTL - and how to create environs that foster intrinsic motivationUndertake individual faculty projectsA goal at Middlesex is to link faculty projects to intrinsic motivation in students. Professor Donna Duffy (psychology), also a Carnegie Scholar, provided information about the Middlesex SOTL initiative.Donna’s own project at Middlesex is ‘Resilience as a Path to Integration in Abnormal Psychology.” She chose resilience, a positive attribute, as a theme because abnormal psychology is generally about negatives … the problems people face. to DonnaWhat’s common to all of the seven campus examples (including Indiana University)?-they involve institutional investment and support for the SOTL program-they are connected to AAHE/Carnegie national initiatives.-they involve faculty investigative projects into issues of teaching and learning
58Task E: Campus Examples of SOTL Initiatives List features of the examples you have just seen that might be most applicable to your campus.Please reflect on the applicability of the campus examples just discussed to your own institution. Construct a “mock-up” of a SOTL program on your campus, drawing on the features of programs at other campuses. Press the button on the frame and send your model to us..
59Unit 2B Faculty SOTL Projects Donetta Cothran – KinesiologyValerie O’Loughlin – Medical SciencesDavid Pace – HistoryWilliam Becker - EconomicsRita Naremore – Speech and Hearing SciencesDennis Jacobs – ChemistryLeah Savion – PhilosophyOther Examples – Carnegie ScholarsTask F: Minute PaperAfter viewing these projects, you’ll be asked to identify the most important points for you in the first four modules of the tutorial (Task F)
60Expectations and Effects of Graded Writing Assignments For Donetta Cothran, a junior faculty member, interest in improving her students’ sense of fairness in grading of their papers led naturally into a scholarly project.Donetta gained insights about scholarship of teaching from her participation in a campus SOTL initiative and involvement in her own scholarly project.to DonettaDonetta Cothran - Kinesiology
61Valerie Dean O’Loughlin Medical Sciences A Neophyte’s Adventures in the Scholarship of TeachingFostering Interactive Learning in a Large Science Course and Methodically Measuring the EffectsValerie’s innate interest in teaching found form and expression in the campus SOTL initiative. Her project is a “Child of SOTL”, conceived, born, and nurtured to scholarly product within the campus initiative.“Scaffolding” facilitated her acquisition of the concepts and resources needed to carry out her study. Valerie’s initial presentation of her work to peers was of interest to many both for the pedagogical techniques she employed and for her means of assessing their effects.Valerie describes the launch of her projectPowerpoint presentation of the complete studyto ValerieValerie Dean O’Loughlin Medical SciencesDetails of Valerie O’Loughlin’s study are considered later in the tutorial in Questions, Designs, and Resources.
62What do I want my Students to be Able to Do? David Pace, a Carnegie Scholar of Teaching, describes his odyssey to the question at the heart of his scholarship; “What are students supposed to be doing (or learning) in my course?” He shares his realization that the above question led to understanding of his discipline in ways he never understood it before.to DavidDavid Pace–History
63William Becker Economics What Does the Quantitative Research Literature Really Show about Teaching Methods?Bill Becker’s project is an example of a study that is not centered on one’s own classes. Rather by studying selected published papers of others, he seeks to separate empirical results from conjecture about the student outcomes associated with classroom assessment techniques and other teaching strategies designed to engage students actively in the learning process.In this project, Bill also seeks to advance the scholarship of teaching and learning by establishing an eleven-point set of criteria which faculty members can use to evaluate inferential studies.Bill’s study culminated in a paper: What Does the Quantitative Research Literature Really Show about Teaching Methods?to BillWilliam Becker Economics1
64A Departmental Level SOTL Project The problem: a group of faculty members concerned that teaching excellence was getting short shrift in the faculty evaluation processThe initial question: What can (and do) student evaluations really tell us?The department had a culture that valued excellent teaching, and in some ways took it for granted. Salary increases were based on data provided using the campus faculty annual report form It had become abundantly clear that those faculty members engaging in innovative or particularly effective pedagogical change were not being rewarded because nobody knew what they were doing or how effective it might be.Rita introduces the projectPowerpoint presentation of complete studyRita NaremoreSpeech and Hearing Sciencesto Rita
65A Departmental Level SOTL Project Issue # 2: Peer Evaluation Once the question regarding student evaluations seemed answered, attention turned to peer evaluation.The faculty consensus: traditional classroom visitation is not very revealing of anything useful. What we really need is evaluation of course structure and materials used in the class for teaching and evaluation. We’d like to have periodic external peer review of these materials.Peer evaluation presented the group with a much larger problem. After much heated discussion, it became clear that the occasional visit to a classroom by one’s colleagues did not seem an adequate form of peer evaluation. This faculty agreed that what was missing was a systematic review of course organization and materials (including exams, visual support for class presentations, etc.) conducted by people outside the department who knew the content.
66A Departmental Level SOTL Project: Ongoing Concerns with Peer Evaluation What are we doing? We’re working toward the development of course portfolios, to be done over time and turned in by faculty members every second or third year, not every year.The biggest concern: IT TAKES TOO LONG TO PUT ALL THIS STUFF TOGETHER!Although agreement in principle was quickly attained, agreement in practice was not. Several faculty members remained convinced that they did not have time to put together their teaching materials in a way that made them accessible to an outside reviewer, especially if they had to explain what they were trying to do. Nonetheless, the department is continuing to work toward a system of course portfolios, along with a checklist that can be used by outside reviewers who are looking at the portfolios.
67A Departmental Level SOTL Project: The Campus and National Ties The departmental course portfolio effort should eventually tie in with a campus-wide effort now underway, conducted by an interdisciplinary team in conjunction with a national project funded by Pew, based at the University of Nebraska, and involving four other research universities.Review the Relationship of Peer Review of Teaching, Course Portfolios, and SOTLVisit the Indiana Campus Course Portfolio InitiativeMore Information About the Consortium Project Led by Nebraskato SimonSimon Brassell Geological Sciences Chair of the Campus Course Portfolio InitiativeThe connection to course portfolio projects exemplifies the way SOTL may be synergistically connected to and integrated with other teaching-related initiatives.
68An Exemplary Course Portfolio And A Superb Model of SOTL An Alternative Approach to General ChemistryAssessing the Needs of ‘At-Risk’ Students with Cooperative Learning StrategiesDennis JacobsProfessor of ChemistryUniversity of Notre Dameto DennisThe next 7 frames contain details of this project. The project exemplifies further the tie between SOTL and course portfolios as well as providing an excellent example of each! Dennis Jacobs is a Carnegie Scholar of Teaching and Learning.The project in complete detail (log-in required)Skip to the next SOTL project.
69Rationale: Recognized Problems: Alternate Course Design: An Alternative Approach to General Chemistry Assessing the Needs of At-Risk Students with Cooperative Learning Strategies Dennis Jacobs (University of Notre Dame)Rationale:Recognized Problems:‘At-risk’ students (Math SAT ≤630):dropped out of General Chemistry.didn’t take any advanced science.frustrated by large lecture format.Alternate Course Design:Similar requirements and lectures.Comparable exams.Various activities involving structured cooperative learning.Initial Comments on Proposal“Only delaying inevitable failure.”“Efforts should be focused on the best not the ‘at-risk’ students.”Comments that colleagues initially made to Dennis are shown in the last bullets. Dennis proved them wrong. He demonstrated convincingly that ‘at risk’ students could perform at the same standards as other students in general chemistry and go on to succeed in organic chemistry (a sequel course) at the same rate.- 4 -
70An Alternative Approach to General Chemistry Assessing the Needs of At-Risk Students with Cooperative Learning Strategies Dennis Jacobs (University of Notre Dame)Documentation:‘At-risk’ students markedly more likely to drop or fail the course in the traditional class format.As you view this project,, note Dennis’ methodologies and the thoroughness of assessment. This is a well designed deed of scholarship. Ask yourself at the end, whether you feel convinced from this brief slide series of the impact of his alternative course for ‘at risk’ students.- 5 -
71Implementation: Aim: Alternate Course Design: An Alternative Approach to General Chemistry Assessing the Needs of At-Risk Students with Cooperative Learning Strategies Dennis Jacobs (University of Notre Dame)Implementation:Aim:Provide improved learning opportunities for ‘at-risk’ students.Develop more effective teaching in large lecture format.Alternate Course Design:Introduced opportunities for structured cooperative learning including:discussion of concepts in pairs.small group in recitation sections.work as pairs in laboratory.Mandatory recitation sections:more time committed to class.direct contact with instructors.- 6 -
72Traditional vs. Alternative Classes: An Alternative Approach to General Chemistry Assessing the Needs of At-Risk Students with Cooperative Learning Strategies Dennis Jacobs (University of Notre Dame)Traditional vs. Alternative Classes:Similarities:Size (250 students), text, chapters.Lecture time (3 hr), lab time (2.5 hr).Lecture format (Powerpoint slides and demonstrations).Exam format and many exam questions.Differences in Alternative Section:Mandatory recitations (1 hr/wk, 20 students); attendance 95% vs. 10%.Weekly homework (10 vs. 30, graded).On-line quizzes (www chapter reviews).Weekly feedback from homework, group problems, on-line quiz, in-class questions.Personal contact with instructor and follow-up if performance declined.- 7 -
73Impact of Alternative Section: An Alternative Approach to General Chemistry Assessing the Needs of At-Risk Students with Cooperative Learning Strategies Dennis Jacobs (University of Notre Dame)Impact of Alternative Section:Assessment Strategies:Effects on conceptual understanding, problem-solving and self-confidence:feedback from students.evaluation of individual elements of the cooperative learning activities.Immediate and long-term benefits:retention of ‘at-risk’ students.success in advanced science classes.Data Collection:Recording in-class learning activities.Tracking individual grades and progress.Longitudinal study of ‘at-risk’ students:progress in subsequent classes.- 8 -
74Measurement of Impact: An Alternative Approach to General Chemistry Assessing the Needs of At-Risk Students with Cooperative Learning Strategies Dennis Jacobs (University of Notre Dame)Measurement of Impact:Success of ‘At-risk’ Students:Better grades in General Chemistry.Improved retention in class.Higher success rate in subsequent classes.- 9 -
75Longitudinal Impact (follow-on course): An Alternative Approach to General Chemistry Assessing the Needs of At-Risk Students with Cooperative Learning Strategies Dennis Jacobs (University of Notre Dame)Longitudinal Impact (follow-on course):Intervention introduced between 1996 and 1997Organic chemistry is a follow–on course to general chemistry. In 1997, a higher percentage of ‘at risk’ students who had taken Dennis’ alternative general chemistry course completed organic chemistry than the percentage of students in the traditional course.- 11 -
76Investigating Pet Theories and Naïve Misconceptions Leah Savion’s personal interest in learners’ cognition and interference effects of naïve misconceptions caught on with faculty colleagues (the issue crosses most disciplines) and led to a cross-disciplinary study.Leah explains misconceptions and their effectsA brief overview may be gained from a one page summary of the projectAdd student misconceptions from your own experience to Leah’s Incomplete Inventory of MisconceptionsFurther detail may be gained from her working paperLeah warmly invites your peer review and comment ( to Leah)Leah Savion Philosophy“What force acts on a coin that has been tossed up in the air”? The large majority of students who completed a course in mechanics in MIT gave the same wrong answer as the totally untrained students, citing the “original upward force of the hand”. The pet theory of motion at work here is that there’s no motion without force. Pet theories are involuntary explanatory constructs that we all build from a very early age, in an attempt to understand the world around us, to build causal connections between events, and to enhance our sense of control over the environment. These theories are about every aspect of life. They are amazingly universal, are based on surface features of the relevant data, fragmented, inconsistent, contain principles that emerge spontaneously, complex, intricate, serviceable, and seemingly well organized. These robust theories are not normally tested against scientific, social or logical facts. A specialized psychological architecture that contains principles of quick reasoning, rapid accumulation of information, and economy of cognitive operations, may underlie and guide their construction and their perseverance in face of contradictory evidence. Pet theories inevitably imply what I termed here “naïve misconceptions”, that prove extremely resistant to change. When the naïve meets the academically acceptable theories we deliver in class, the better students either attempt to alter the new information, or combine the incompatible principles, or, most commonly, adapt the new theory as a “school bound” explanation, not applicable in everyday life.
77Scholarly Projects of Faculty (Continued) Opening Lines may be a useful source for identifying issues of teaching and learning in one’s practice and framing them. Many faculty will be able to identify with the thoughts and experiences of the authors of these case studies. For faculty members who feel that they do their “thinking about teaching” in isolation within their departments or institutions, this publication may feel like a conversation with colleagues.Where you can learn about additional scholarly projects. Opening Lines contains 8 case studies of approaches to individual projects by Carnegie Scholars. The seven bullets below form the outline of the narrative in each case study. Style and tone are similar to those in the video vignettes used in this tutorial.Framing the Question- Context of the Project- Gathering the Evidence- Emergent Findings- Conditions for doing SOTL- Benefits of the Work- Lessons LearnedOpening Lines comes with a CD containing a variety of tools and resources for scholarly projects. These include interviews (some as video clips), survey instruments, protocols, and reference sources. The publication showcases recent work in CASTL
78Task F: Looking Back Over Travel Along the Continuum Thus Far "MINUTE PAPER"1. What were the 3 most important points presented in Units 1A, 1B, 2A and 2B of this tutorial?2. What 3 things would you most like to learn in the remaining two modules, Units 3A and 3B, of the tutorial?.This Classroom Assessment Technique (CAT) denotes the end of Unit 2B. The formative feedback offered by you to us here will help make this a better tutorial for everyone. Please press the button and send your feedback. If you’ve worked through the tutorial continuously to this point, you might want to take a break before entering Unit 3 - the last stretch of the continuum from alpha to omega (and it’s more than a third)!
79Unit 3A Bridges to Productivity How Could I Do Scholarship of Teaching and Learning (Genres of SOTL)Task G: Reflecting on GenresApproaches to Scholarship Via Classroom Research ProjectsWhere to Publish and PresentSources of External FundingTask H: Reflecting on Classroom Research and Key Reference BooksUnit 3 may be thought of as a rudimentary “tool kit” for embarking on scholarly projects in issues of teaching and learning. This is the longest unit. 3A surveys some general considerations. 3B focuses on project design.
80Still Another Look at What We Mean by the Scholarship of Teaching and Learning Before venturing into a mélange of categorization schemes, it may be well to pause for another overview of this scholarship.Do the following situations represent scholarship of teaching and learning?A faculty member reforms a course, building into the reform all three criteria of scholarly teaching. She puts the course on-line not only for students but for all faculty in the discipline, and furthermore invites faculty to peer review and critique it on the web. The course catches the attention of faculty at another institution. They are interested in it and invite the course author to come there and give a colloquium on the course. The author does so and reflects on the attributes of the course for her students. Subsequently, the department in the second institution adopts large elements of the author’s course to improve their own undergraduate program.An assistant professor in quest of tenure develops a course portfolio as evidence of his effectiveness in improving student learning. The promotion and tenure committee routinely sends candidate packages to outside reviewers. External reviewers comment on the excellence of this portfolio as convincing evidence of the candidate’s teaching effectiveness. The candidate is granted tenure. The course portfolio is subsequently disseminated through the state university system as a model for other faculty.
81Still Another Look at What We Mean by the Scholarship of Teaching and Learning (Cont.) If you accept Shulman’s quasi-definition , you will probably conclude that both situations in the previous frame DO represent scholarship because they satisfy all three of his conditions. As key reformers urged, the concept of scholarship is broadened to include useful contributions that do not necessarily conform to conventional scholarly forms in the disciplines. To be sure, scholarship of teaching and learning includes publication in journals, but it also includes other forms, as in the two previous examples, that contribute to the advancement of teaching and learning both directly and indirectly.To survey possibilities for this scholarship, we exhibit several categorization schemes. First, we’ll consider Craig Nelson’s “Genres of SOTL.” Then we’ll look at “traditional and classroom research” and “qualitative and quantitative methods.” All of these schemes are messy and inconsistent. The genres, for example, sometimes seem separated by unit of analysis at other times by type of research design. And as we see from the two examples in the preceding frame, they are not necessarily inclusive of all possible forms of scholarship. Similarly, much good scholarship combines elements of traditional and classroom research as well as qualitative and quantitative methods.Well, why do we even bother with the classification schemes if they are inconsistent? Because they give us valuable “points of possibility.” They may be thought of as topographical features that map the territory. The actual point at which one locates one’s individual deed of scholarship is usually somewhere between the “points of possibility.”
82How Could I do Scholarship of Teaching and Learning? Craig Nelson introduces the genres of SOTLDifferent Genres of SOTLReports on Particular ClassesReflections on Years of TeachingLarger Contexts—ComparisonsFormal ResearchMeta-AnalysesGenres of SOTL overlap and can be combined or subdivided variously. The particular examples of scholarship for each genre are chosen to illustrate importance for improving teaching and learning and largely excerpted from Craig Nelson’s paper on genres: "How could I do scholarship of teaching and learning?"Two opening points:1. Learning and teaching are complex activities where approximate, suggestive knowledge can be very helpful, and, indeed, may often be the only kind that is practical or possible2. Much important expertise on teaching resides in the day to day practices of good faculty. Typically, this knowledge remains private and is totally lost when its possessor retires. A key task in this field is systematically making much more of this expertise public.
83Reports on Particular Classes How Could I do Scholarship of Teaching and Learning?Different Genres of SOTLReports on Particular ClassesReflections on Years of TeachingLarger Contexts—ComparisonsFormal ResearchMeta-AnalysesReports on Particular ClassesIt Worked! Important pieces of our expert knowledge as experienced practitioners can be preserved by writing up exemplary approaches to content or pedagogy that work especially well in our own classes. In this genre, the teacher's own impressions of the effectiveness frequently serve as sufficient assessment. The trend now is to try to document the effectiveness a bit more formally using CATs and CR.Before & After: Assessments of Changes in PracticeQualitative The many examples of this genre in Angelo and Cross include a calculus class (pp ) in which the professor wanted to help students improve their problem solving skills. This example illustrates the process of refining the pedagogical questions and the successive modifications that are often necessary to make new pedagogical approaches work successfully. In this case, the new pedagogy improved student success sufficiently that no student made an F, despite the maintenance of high academic standardsQuantitative Group study approaches to calculus decreased the D/F/W rate for African Americans from 60% to 4%. See R. E. Fullilove & P. U. Treisman Mathematics Achievement Among African American Undergraduates at the University of California, Berkeley: An Evaluation of the Mathematics Workshop Program. Journal of Negro Education 59:
84Reflections on Years of Teaching How Could I do Scholarship of Teaching and Learning?Different Genres of SOTLReports on Particular ClassesReflections on Years of TeachingLarger Contexts—ComparisonsFormal ResearchMeta-AnalysesReflections on Years of TeachingEssays Developing Good Ideas An example providing good articulation of the rationale for SOTL is L. S. Shulman Teaching as community property: Putting an end to pedagogical solitude. Change 25: 6-7.Summaries of Expert Knowledge Gained by Self-Reflection and Experimentation Examples are P. Frederick The Dreaded Discussion: Ten Ways To Start. Improving College & University Teaching 29: P. J. Frederick The Lively Lecture--Eight Variations. College Teaching 34:43-50 Many course portfolios fit here too. Path breaking examples can be accessed through links at Implementing Entities: Carnegie and Pew. Another example by Randy Bass is (http://www.georgetown.edu/bassr/portfolio/amlit/ .Integration of Larger Frameworks with Classroom and Curriculum Practice J. D. Herron Piaget for Chemists: Explaining What "Good" Students Cannot Understand. Journal Chemical Education 52: One factor that explains why bright, hard-working students can do poorly and how we can help them. Easily applicable in all quantitative fields. R. J. Kloss A nudge is best: Helping students through the Perry scheme of intellectual development. College Teaching 42: Another factor that explains why bright, hard-working students can do poorly and how we can help them. Easily applicable across the curriculum.
85How Could I do Scholarship of Teaching and Learning? Different Genres of SOTLReports on Particular ClassesReflections on Years of TeachingLarger Contexts—ComparisonsFormal ResearchMeta-AnalysesLarger Contexts: Comparisons Across Courses and Student Change Over TimeQualitative Studies to Explore a key Issue Wm. G. Perry, Jr.  Forms of Intellectual and Ethical Development in the College Years, A Scheme. New introduction by Lee Knefelkamp. Jossey-Bass. The impetus here was the observation that students could flunk out of Harvard despite working quite hard at learning the course material. The longitudinal design used extensive interviews with students at the end of each of their four undergraduate years. Patterns of intellectual development were inferred and checked for inter-judge reliability. A very influential study.Quantitative comparisons of Different Courses or Sections M. D. Sundberg & M. L. Dini Science majors vs nonmajors: Is there a difference? Journal of College Science Teaching. Mar/Apr 1993: Question: Does covering more teach more?. Both courses taught with traditional pedagogy and by multiple instructors, but with different intensities of 'coverage.' Learning assessed with the ACT exam for AP Biology "The most surprising, in fact shocking, result of our study was that the majors completing their course did not perform significantly better than the corresponding cohort of nonmajors."Comparisons over an Array of Courses with a Common Assessment Instrument R. R. Hake Interactive-engagement vs traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses. American Journal of Physics 66: (http://carini.physics.indiana.edu/SDI/welcome.html#z44) Uses a common test of Newtonian physics to compare increases in understanding achieved by a wide array of pedagogies in introductory physics courses at institutions ranging from high-schools to Harvard. Found that "interactive engagement" approximately doubles the amount of physics earned. An especially important model for emulation in other disciplines.
86Formal Research How Could I do Scholarship of Teaching and Learning? Different Genres of SOTLReports on Particular ClassesReflections on Years of TeachingLarger Contexts—ComparisonsFormal ResearchMeta-AnalysesFormal ResearchWhat constitutes “formal”? Does “formal” reside in the lower right quadrant of Rice’s diagram? Not necessarily. We use “formal research” to mean forms of scholarship in which the principal goal is research. Usually such scholarship will incorporate one or more features of conventional research paradigms. Formal research can still be classroom-based and related to active practice (see the last three examples below).Experimental Analyses C. M. Steele A threat in the air: How stereotypes shape intellectual identity and performance. American Psychologist 52:Classroom Incivilities R. Boice Seminal study on incivilities. Identifies and counts perceived incivilities committed by faculty as well as students and investigates relation between them. Journal of Research in Higher Education 37:The Impact of One Minute Papers on Learning in an Introductory Accounting Course E. D. Almer, K. Jones, & C. Moeckel Four hypotheses investigated simultaneously in one class using a fractional factorial design. Issues in Accounting Education 13:The One-Minute Paper: Some Empirical Findings J.F. Chizmar & A. L. Ostrosky Four teachers each taught control and experimental sections. Data were analyzed using a multiple regression model. Journal of Economic Education 29: This study like the one above on incivilities could also be categorized under “Larger Contexts” The last two studies of one-minute papers are good examples of formal research at the classroom teacher level. They are also important contributions to the structure of evidence that one-minute papers improve student learning.
87How Could I do Scholarship of Teaching and Learning? Different Genres of SOTLReports on Particular ClassesReflections on Years of TeachingLarger Contexts—ComparisonsFormal ResearchMeta-AnalysesMeta-AnalysesAnnotated Bibliographies R. N. Johnson, D. M. Enerson & K. M. Plank Diversity: A Selected and Annotated Bibliography Center for Excellence in Learning and Teaching. Pennsylvania State University.Brief Annotated Summaries of Key Research Findings T. A. Angelo The campus as learning community: Seven promising shifts and seven powerful levers. AAHE Bulletin 49:3-6.R. B. Barr & J. Tagg From teaching to learning: A new paradigm for undergraduate education. Change 27:13-25.Formal (Quantitative) Meta-Analyses L. Springer, M.E. Stanne & S.S. Donovan Effects Of Small-Group Learning On Undergraduates In Science, Mathematics, Engineering And Technology, A Meta-Analysis. National Institute for Science Education, University of Wisconsin. 608/ [average effect size "would move a student from the 50th percentile to the 70th..."]
88Task G: Reflecting on Genres Where does the course portfolio fit in the genres described?Reflecting on the genres, which seems most appropriate for a scholarly project of your own? Which seems easiest for you to do?Are there forms of SOTL for which the genres described do not apply? If so, how would you name one or more additional genres?RESPONDING TO THIS TASK IS NOT JUST BUSYWORK! Reflection on the “points of possibility” described in the genres, may be an important step in conceptualizing your own work in SOTL. You may want to read one or more of the example papers before attempting this task. Your conception of your own SOTL work need not coincide with one of the genres, of course!
89Approaches to Scholarship Via Classroom Research Distinguishing classroom research from traditional researchA “grand schematic” of classroom researchClassroom Assessment TechniquesEffective GradingThe Course PortfolioTask H: Reflecting on Classroom ResearchClassroom research is described. A schematic diagram is exhibited and used to show strengths of various useful books. Finally, in Task H you will be asked to identify the step in classroom research that seems easiest and the reference book that seems most useful.
90Classroom Research“Classroom Research is not traditional research conducted in or on classrooms. It is a specific methodology designed for discipline oriented teachers without training or experience in the methods of educational research. Classroom Research is ongoing and cumulative intellectual inquiry by classroom teachers into the nature of teaching and learning in their own classrooms. Inquiry into a question about how students learn typically leads to new questions and thus to continual investigations through classroom research.” Cross & Steadman“Finding that almost no relationship existed between research on learning and collegiate teaching practices (professors were either oblivious to such research or ignored it), K. Patricia Cross concluded that the research itself was at fault for failure to pay attention to actual classrooms. She argued that teaching and learning reforms could only occur if they were based on concrete classroom situations. Cross made “how to do it” her calling card.” M. Lazerson, U Wagener, and N. Shumanis. Teaching and Learning in Higher Education. p. 17 Change 32:Many journals that publish articles about teaching and learning, both within disciplines and across disciplines, invite articles based on classroom research.The term “action research” is sometimes used synonymously with classroom research. The former dates from the work of Kurt Lewin in the 1940’s and may be defined as “Research carried out by practitioners with a view to improving their professional practice and understanding it better.” Borg, Gall & Gall Applying Educational Research 3rd ed. Longman,: 390Cross, K.P. and Steadman, M Classroom Research:Implementing the Scholarship of Teaching, Jossey-Bass
91“Traditional” and “Classroom” Research Traditional and Classroom research are compared for the sake of characterizing two “points of possibility.” One is not advocated over the other and it’s an oversimplification to view them as independent. Traditional research is often performed in classrooms. Classroom research may be enhanced by traditional methods and measures, especially if the results are for publication in national journals. For example, use of an “off the shelf” assessment instrument with widely accepted validity may be preferable to a “home grown” test or survey with unknown validity. Using someone other than the person who determines grades to collect potentially sensitive data from students is also a traditional method that may enhance a classroom research effort (and make it compliant with the local human subjects policy!)TraditionalClassroomOriginState of present educational literatureProfessor’s teaching practicePurposeContribute to existing state of educational theoryObtain practical knowledge applicable in limited circumstancesRequirementSpecialized training / broad grasp of pertinent literatureSpecialized training not essential but developmental scaffolding may beBenefitField and researcherStudents and professorA key point is that classroom research is a good point of embarkation for the typical faculty member using materials at hand and without specialized training in educational research methods. Classroom research is usually directed at local teaching issues or questions of specific interest to the teacher.“Classroom research is not about finding universal truth but rather contextualized truth.” Tom Angelo
92Improving Teaching and Learning via Classroom Research Our grand schematic of classroom research! Schema with similar features are common; We envision three major components to a research project; a motivating goal or issue, acquisition of information bearing on achievement of the goal or resolution of the issue, and effects identifiable from the juxtaposition of impetus and information. Sometimes the project involves an intervention, but not necessarily.INFORMATIONGATHERINGIMPETUSFormativeAssessmentEFFECTGoalSummativeAssessmentAnalysisSCHOLARLYPRODUCTIVITYIssueRecordedObservationReflectionExistingScholarshipSynthesisWe argue that if the scholarly project is done well, teaching and learning will inevitably improve regardless of the results of the study. Why so? Because the classroom research by its very nature will at least improve attention to students and information they provide, thereby improving understanding and communication. Student performance is also positively motivated by evidence that teachers care enough to study them and the efficacy of their learning.InstitutionalInformation BaseAfter the third major component in the above schema (Effect) the possibility of a scholarly product may emerge (curriculum materials, article, presentation, etc.) Such a product may enhance the institution’s information base as well as the existing body of scholarship.APPLICATIONImprovedTeaching andLearning
93Classroom Assessment Classroom assessment is systematic and formative Class is the unit of measurement rather than the individualConditions of learning may be assessed rather than student performance.Correct and incorrect are not the emphasis.Unexpected rather than expected responses are often most useful.Review Unit I material about classroom assessment/researchDefining features of classroom assessment are shown in the bullets.They are quite different from the features of summative assessment associated with traditional grading.The book belowhelps faculty clarify their teaching goals, explains formative assessment to improve achievement of goals and provides a catalog of 50 classroom assessment techniques (CATs).INFORMATIONGATHERINGIMPETUSFormativeAssessmentEFFECTGoalSummativeAssessmentAnalysisSCHOLARLYPRODUCTIVITYIssueRecordedObservationReflectionExistingScholarshipSynthesisInstitutionalInformation BaseAPPLICATIONImprovedTeaching andLearningAngelo, T. & Cross, K.P. (1993) Classroom Assessment Techniques:A Handbook for College Teachers (2nd Ed) Jossey-Bass
94Effective GradingPrimary Trait Analysis (PTA)… building scales that make performance criteria explicit in order to:-Categorize/classify student work.-Benchmark student learning and document changes.-Improve validity of grading.The book below:Is a useful resource for classroom researchersAims at documenting how well students are achieving cognitive objectives and at providing evidence for change in achievement over timeTreats almost any conventional objection or concern about gradingINFORMATIONGATHERINGIMPETUSFormativeAssessmentEFFECTGoalSummativeAssessmentAnalysisSCHOLARLYPRODUCTIVITYIssueRecordedObservationReflectionExistingScholarshipSynthesisInstitutionalInformation BaseAPPLICATIONImprovedTeaching andLearningWalvoord, B. and Anderson, V. (1998) Effective Grading: A Tool for Learning and Assessment, Jossey-Bass .
95The Course Portfolio“I was familiar with teaching portfolios … but thinking about teaching as scholarly inquiry began to lead me in the direction of something I had not seen anyone else doing: a portfolio that focused on the course rather than on all of one’s teaching. Being a social scientist, I began to think of each course … as a kind of laboratory - not a truly controlled experiment of course but as a setting in which you start out with goals for student learning, then you adopt teaching practices that you think will accomplish these and along the way you can watch and see if your practices are helping to accomplish your goals, collecting evidence about effects and impact.”W. Cerbin quoted in Hutchings, P. (Ed.) 1998 The Course Portfolio, AAHEReview Unit I material on course portfoliosReview a course portfolio example.As a kind of laboratory notebook for recording data and observations over time, the course portfolio keeps the teacher’s experience from being lost with the teacher’s memory. It’s a possible means of capturing “wisdom of practice.”
96Where to Publish and Present Venues for publication and presentation of work in SOTL abound. Journals and conferences may be discipline-based or inclusive of all disciplines, refereed or non-refereed, and local, regional, national, or international. Publications may be printed or on-line and conferences may be physical or virtual (teleconference).Many sources of publication are evident in references given throughout the tutorial. Additionally, over 100 journals publishing SOTL are accessible through a list of journals publishing scholarship of teaching and learning.Opportunities to present in cross-disciplinary venues (calls for proposals and conference announcements) are provided by the Professional and Organizational Development (POD) Network in Higher Education and the American Educational Research Association (Division J- Postsecondary Education) Both POD and AERA maintain electronic mail lists (listservers) which include calls for proposals and conference announcements from all over the world. More information about these lists are available at the above links.
97Potential Sources of External Funding Sources of grant funding for specific topics and categories in higher education are generated by search engines operating on SPIN and COS databases. Most institutions subscribe to at least one of these databases. See your local sponsored research services or institutional research services for more information. Some sources, but by no means an exhaustive list, are shown below.Pew Charitable TrustsFund for Improvement of Postsecondary EducationNational Science FoundationWoodrow Wilson National Fellowship FoundationAlfred P. Sloan foundationDepartment of EducationSpencer FoundationAndrew W. Mellon FoundationEli Lilly Foundation
98Task H: Reflecting on Classroom Research Which major step in the classroom research schematic (determining the goal or issue, gathering essential information, or determining effect) seems the easiest for you to carry out? Why?Which of the texts described in this section seems most immediately useful to you?
99Unit 3B Questions, Designs, and Methods Framing QuestionsTask J: Examining Valerie’s QuestionsTask K: Framing Your Own QuestionDesigns for StudiesChoosing MeasuresGuiding Questions in Choosing MethodologyTask L: Designing Your Own ProjectSummary of StandardsTask M: Evaluation of the TutorialWe begin the final module of the tutorial with a disclaimer; the possibilities are too many for adequate survey here. The topics in this module could easily fill a textbook (and, in fact, do fill many textbooks!) We limit coverage to a few possibilities.
100Framing the QuestionScholarship of teaching and learning does not necessarily begin with a question. To record “what worked,” the “wisdom of experience” or to describe observed phenomena without attempting intervention can constitute scholarship. To integrate the work of others, as in constructing a continuum of SOTL experience from Alpha to Omega, or as in editing a collection of papers into a book might also constitute scholarship. Nevertheless, much scholarship is rooted in inquiry because research is rooted in inquiry.Faculty members do not necessarily receive formal preparation for teaching, much less scholarship of teaching. Therefore, to expect faculty members to know how to transform their issues and goals of teaching and learning into researchable questions without a little tutoring does not seem quite reasonable.As many dissertation advisors and doctoral students will attest, the most difficult step in the whole research process is often framing the question to be addressed. A well framed question is one for which procedures can be devised that offer the possibility of arriving at an answer.The next series of frames are rather narrowly focused on this particular challenge.
101Framing Questions (Goal Approach) Define a goalAsk and answer questions to refine the goal (Colleagues are usually helpful.)Create a succinct summary of a specified goalAsk “what evidence would reveal that the goal is achieved?”Frame possible researchable questionsTeachers commonly think of goals in terms of improved learning outcomes for students. Often, a study is built upon one or more interventions to achieve a goal and the question becomes “Does the intervention lead to achievement of the goal?” The goal approach also includes inquiries which Pat Hutchings might characterize as “What works?” However, a study based on a goal approach need not involve an intervention. A well-framed question could simply address whether an intended goal is presently being achieved or whether a goal is even feasible to achieve.
102Framing Questions (Issue Approach) Criteria for selection of issuesInvestigable (not necessarily empirical)Bounded and well-definedSignificant (not necessarily statistically)Considerations for investigation of issuesLength of time neededComplexity of proceduresAvailability of subjectsAvailability of support (resources, personnel, funds)Questions can involve investigation of issues rather than achievement of goals; e.g. “How do students who do not meet prerequisites fare compared to those who do?” In Opening Lines, Mills Kelly describes how his department chair framed Mills’ question by asking, “How do you know that using the Web as opposed to depending on paper (the way most of us have taught history) is transforming student learning – and, if so, whether for good or for ill?” Other types of questions like “What does the current learning environment for students in my course look like to them?” , “How do they perceive my instruction?” or “How do students who yield evidence of deep understanding in my course, gain that level of understanding?” can lead to investigative studies.
103Making Vague Questions Answerable – Using Operational Definitions Vague questions are made answerable by using operational definitions of terms to improve conciseness. Operational definitions sharpen the question and afford possibility of clear answers. The price paid is usually a sacrifice in generality of answers obtained.Less framedDo students who help others learn an academic discipline learn it better themselves?Do students learn more in small classes?What is the optimum number of homework assignments to give in a beginning math class?More framedDo students in CMSC 250 who tutor students in CMSC 150 perform better on the CMSC 250 final exam than students who do not tutor but have similar grades in CMSC 150?Do students in sections of Phys 118 enrolling fewer than 50 students perform better on the departmental final exam than students from sections enrolling more than 75 students?Do students enrolled in M036 who are given a homework assignment every week perform differently on the departmental final exam than students enrolled in M036 who are given homework every class period?Considering small classes to be less than 50 students and “learning” to be characterized by final exam performance as in the above example are arguable definitions, of course. Nevertheless these operational definitions afford obvious and necessary clarity to the structure of the study.
104Task J: Examining Valerie’s Questions A Neophyte’s Adventures in the Scholarship of TeachingTask J: Examining Valerie’s QuestionsReview the introduction of Valerie’s projectValerie’s QuestionsWill the use of interactive learning activities in A215 lecture improve exam performance?Will these activities improve lecture attendance and create a more active student engagement in lecture?3. Will these activities improve overall “merit success rate” in A215?Valerie Dean O’Loughlin Medical SciencesValerie’s questions need further specification. What terms , in particular, need improved operational definition?After submitting your answers, compare them with the answers in the following frame.
105A Neophyte’s Adventures in the Scholarship of Teaching Task J (Answers)Terms particularly needing improved operational definition:“interactive learning activities”Primary intervention: use of one or more interactive learning activities per lecture sessionExamples of Learning Activities*Memory MatricesLearning ExercisesSample Exam Questions“Muddiest Point”“active student engagement”Valerie defines this term orally (beginning at 1 minute: 40 seconds into the video clip)“merit success rate”merit success rate = total A,B,C’s initial enrollment* Angelo, T and KP Cross (1993). Classroom Assessment Techniques: A Handbook for College Teachers. 2nd Ed. San Francisco: Jossey-Bass
106Framing Questions as Hypotheses H1: Students who write one-minute papers will perform better on a subsequent a. essay b. multiple choice quiz than students who do not write one-minute papers.H3: Students whose one-minute papers are graded will perform better on a subsequent quiz than students whose one-minute papers are not graded.H1 actually contains two hypotheses. Why?H4: The difference in quiz performance between students who do and do not write one minute papers will be less for higher ability students than for lower ability students.H2: Students who address their one-minute papers to a novice audience will perform better on a subsequent quiz than students who address their papers to the instructor.In some studies, questions are most usefully framed as hypotheses to be tested. All of these hypotheses were successfully tested in the referenced study.Almer, E.D., Jones, K., & Moeckel, C.L. (1998) “The Impact of One-Minute Papers on Learning in an Introductory Accounting Course.” Issues in Accounting Education (13) 3
107Task K: Framing Your Question Write a tentative question to be addressed in one of the courses you teach.Discuss your question with one or two colleagues for the purpose of framing it in the clearest and most answerable way. Encourage your colleagues to challenge your framing.Write your well-framed question at the end of this process.THIS TASK MAY TAKE YOU AWHILE. You might want to stop here, think about your question, take it down the hall for discussion with colleagues, and bring back the results after lunch. You might also want to compare and contrast it to the questions in Faculty SOTL Projects before writing it in well-framed form and pressing the button to send it.
108What is a Design for a Study? A plan or protocol for carrying out the studyAn underlying scheme that governs functioning, developing, or unfoldingA good design always promotes efficient and successful gathering and analysis of the needed information.Tthe design makes clear what information is being gathered (Example: student perception by means of fixed interview procedure in focus groups) or is being measured (Example: student learning by means of 3 exams). The design also makes clear what, if anything, is being manipulated. (Example: the study is designed so that some students attend review sessions before each exam while others do not.)The nature and frequency of measurement and manipulation are made clear along with factors that the investigator is attempting to take into account because they may explain the result even though they are not the focus of the study. (Examples of such factors are students’ class attendance or backgrounds.)The design should include a timetable for significant milestones, specification of measures to be used in acquiring information, and provision for obtaining needed resources (Example of milestone: institutional approval to use your students as subjects. Two examples of needed resources: grade distributions of students in the target course over the last several years; a colleague to conduct interviews of your students.)It may be necessary to modify the design as work progresses. In general, the greater the degree of qualitative information gathering, the more likely is modification in the design as the study progresses. It’s important that any design modification be the result of careful consideration rather than something that just happens in the course of events without the implications for the goals of the project having been thought through.
109A Qualitative or a Quantitative Study A Qualitative or a Quantitative Study? Danger: This may not be the best question to ask!Why may the above question not be the best one to ask? Because it often tends to polarize faculty along their disciplinary lines and this may not be in the best interest of excellent scholarship of teaching and learning!No factor is more influential in shaping our teaching and research perspectives than our individual disciplines. For most of us, almost our every word and thought about either teaching or research is grounded in our experience in the discipline. The research methods traditional in a discipline tend to influence the approaches faculty in that discipline take toward SOTL (and what journals in that discipline will accept for publication as SOTL).An interesting experiment is to pose an issue for investigation to a cross-disciplinary group of faculty members. If encouraged to talk about approaches to the investigation, their approaches might be very different, reflecting their different disciplinary homes. Often each approach has potential to shed light on the issue. The scholarship that is most effective in advancing teaching and learning may be that which combines a variety of approaches.But there is a danger. Rather than achieving synergy between different approaches, we sometimes tend to discredit or reject approaches which are not familiar to us.I felt I was moving between two groups … who had almost ceased to communicate at all, who in intellectual, moral and psychological climate had so little in common … literary intellectuals at one pole – at the other scientists …Between the two a gulf of mutual incomprehension – sometimes hostility and dislike, but most of all lack of understanding. They have a curious distorted image of each other. Their attitudes are so different that even on the level of emotion, they can’t find much common ground C. P. Snow**Snow, C. P. (1959) The Two Cultures and the Scientific Revolution. Cambridge University Press pp.2-4
110Quantitative and Qualitative Methods as “Points of Possibility” Featurenaturegoaldesignsamplemeasuresinvestigatoranalysis methodfindingsQuantitativeempirical, statistical, comparativehypothesis testing, confirmatorypredetermined, fixedlarge, representativescores, percentages, counts, ratesoutsider, non-perturbingdeductivesummative, precise, reliableQualitativenaturalistic, fieldwork, constructivistdescriptive, generative, finding meaningflexible, evolvingsmall, purposefulinterviews, observations, writingsinsider, perturbinginductiveformative, rich, expansiveAs in previous comparisons, we do not “go to the wall” in defense of the absolute correctness or comprehensiveness of the above table entries. Nevertheless, they are offered as generally useful characterizations.
111Typical Measures Associated with Quantitative and Qualitative Methods course exam, project, paper scoressurvey scores (Likert)frequencies of multiple choice test item responsesscores on standardized scales and testscounts (participation, web requests, office visits)measures of time useinstitutional research data (GPAs, grades, admissions scores, frequency distributions of demographics)Qualitativeperformances (possibly recorded on tape)interviews (possibly recorded on tape)focus groupsstudent projects, term papers, essay items or examsreflective statementsjournalsreports of others (counselors, etc.)In general, the more ways of gathering information and the more frequent the events in which information is gathered, the greater the validity of the study.
112Examples of Measures in a Particular SOTL Project A Neophyte’s Adventures in the Scholarship of TeachingExamples of Measures in a Particular SOTL ProjectReview the introduction to Valerie O’Loughlin’s projectReview Valerie’s research questionsValerie discusses content of the box belowAssessment Measures“Affective” measuresmuddiest point, mid-course survey“Process” measureslecture attendance, Web hits/requests on learning activities“Performance” measures (for both the Class and the Instructor)Fall 2000 compared to Fall 99, 98, 97 semestersIn the next 6 frames of the tutorial, some specific measures in Valerie’s study and the information gained from them is surveyed.Skip over the examples of measures from Valerie’s study.
113A Neophyte’s Adventures in the Scholarship of Teaching Examples of Measures in a Particular SOTL Project: Mid-Semester EvaluationThe mid-semester evaluation was a questionnaire designed by Valerie that students completed individually or in groups during class time and submitted anonymously. Key findings were:Lecture notes on Web very helpfulMore learning activities requestedExtra lecture review sessions requestedReview sheets requestedWanted more exams that cover less materialThe second finding confirmed that students liked the learning activities (which were the primary intervention in the study) and wanted more of them. Valerie granted the requests in 2, 3, and 4, during the semester of her study and is considering 5 for future semesters.
114A Neophyte’s Adventures in the Scholarship of Teaching Examples of Measures in a Particular SOTL Project: Count of Web Hits on Learning ActivitiesAre students really accessing the learning activities?Lecture Requests:Integument: 315Myology: 312Muscles of Upper Limb: 311Nervous System: 315Learning Activity Requests:Epithelium: 237Myology 1: 159Muscles of Upper Limb: 155Neurons: 136Web hits were counted as a measure of student use of learning activities outside of class. The left and right columns above respectively show number of requests for lecture notes and the associated learning activity during a particular month (September). The data below show that overall web hits associated with the target course increased sharply in the semester in which the learning activities were implemented.Average Number of web requests per student:Fall 1998: Fall 2000: 176
115A Neophyte’s Adventures in the Scholarship of Teaching Examples of Measures in a Particular SOTL Project: Pre-course data comparisonsSAT-VSAT-MGPACredit Hours19975245342.984719985335453.044819995182.9546200051653649Grand Avg.5372.99The purpose of background measures is to investigate whether students across semesters are “fairly similar.” This controls for the possibility that any improvement shown on outcome measures in the study is because the class using the intervention consisted of higher achievers than the comparison group. In this case, the higher achievers appeared to be in the 1998 class rather than the class involved in the intervention (2000).
116Examples of Measures in a Particular SOTL Project: Common Exam Item A Neophyte’s Adventures in the Scholarship of TeachingExamples of Measures in a Particular SOTL Project: Common Exam ItemMotor neurons are examples of ______ neurons.BipolarAfferentAssociationPseudounipolarMultipolar*A common exam item can serve as a measure of learning. Except for minor changes in distractors, this item was the same over four years of tests in the target course. In the SOTL project (2000), a learning activity about types of neurons was done in class.
117A Neophyte’s Adventures in the Scholarship of Teaching Examples of Measures in a Particular SOTL Project: Mean Exam PerformanceExam 1Exam 2Exam 3Exam 4200080.375.379.681.2199974.472.576.576.9199871.671.87882.4199774.769.87578.3Exam scores are frequently used as measures of learning. For a comparison of exam scores across years to have any meaning, the comparability of the exams across the years must be demonstrable. The comparability of exams was demonstrable in this study and students scored higher on 3 of 4 exams during the semester (Fall 2000) in which the learning activities were employed.
118A Neophyte’s Adventures in the Scholarship of Teaching Examples of Measures in a Particular SOTL Project: Instructor EvaluationsFall 2000 notables:-highest overall mean evaluation-highest score for question “my instructor clears up points of confusion for me”-highest score for question “my instructor facilitates discussion among students”“The format of lectures, notes and learning exercises appealed to my style of learning”“She made learning easier by using different teaching techniques.”“She gave more personal attention in a class of 250+ than many do with much smaller classes. She consistently made us feel that she wanted us to succeed and that she would go the extra distance to make that happen.”“Dr. O’Loughlin is an excellent teacher. I speak as a humanities student who might turn to science if every science instructor were like her.”The instructor evaluation is commonly used as both a quantitative and a qualitative measure. An end-of-course evaluation alone may not make a very compelling body of evidence in a study. However, when the evaluation is used in conjunction with a variety of quantitative and qualitative measures as in this SOTL project, a clear and compelling work of scholarship may result.
119Guiding Questions in Choosing Methodology What approach fits your research problem?Qualitative case studyQuantitative study enhanced by qualitative dataQualitative study enhanced by quantitative dataDo you have the skills/resources to carry out the methods?The skills/resources needed to employ qualitative and quantitative methods appropriately tend to be very different. Pooling of skills and resources is a good reason for community and collaboration in a campus SOTL initiative.Will your audience find these approaches acceptable?Editors or audiences within a discipline may be disinclined to accept work that does not include methods familiar to that discipline.A good primer: User Friendly Handbook for Mixed-Method Evaluations (NSF)Why such a publication from the premier organization representing the sciences? “ Because of the recognition that by focusing primarily on quantitative techniques, evaluators may miss important parts of a story.” Similarly by focusing exclusively on qualitative methods, researchers may miss opportunities to back their findings with the kind of objective evidence that makes the findings credible to a broader audience.Guiding questions provided by Samuel Guskin, Professor Emeritus, School of Education, Indiana University
120Task L: Designing Your SOTL Project Individually design a study to address the question you framed in Task K.Discuss your tentative design with with colleagues (perhaps the same ones who helped refine your question.) Encourage your colleagues to question you and comment.Sketch any revisions to your design at the end of this process.Like Task K, This task may take awhile to complete. Your design may involve sketches, or other graphics and tables or charts. Completing this task is an important step in processing the content of this unit. It is also a critical step if you plan to undertake a SOTL project!
121Effective Presentation Summary of StandardsThe six general standards for scholarship below seem an excellent wrap-up of our work in the tutorial. George Walker comments on standards for SOTLClear GoalsDoes the scholar state the basic purpose of his or her work clearly? Does the scholar define objectives that are realistic and achievable? Does the scholar identify important questions in the field?Adequate PreparationDoes the scholar show an understanding of existing scholarship in the field? Does the scholar bring the necessary skills to his or her work? Does the scholar bring together the resources necessary to move the project forward?Appropriate MethodsDoes the scholar use methods appropriate to the goals? Does the scholar apply effectively the methods selected? Does the scholar modify procedures in response to changing circumstances?Significant ResultsDoes the scholar achieve the goals? Does the scholar’s work add consequentially to the field? Does the scholar’s work open additional areas for further exploration?Effective PresentationDoes the scholar use a suitable style and effective organization to present his or her work? Does the scholar use appropriate forums for communicating work to its intended audiences? Does the scholar present his or her message with clarity and integrity?Reflective CritiqueDoes the scholar critically evaluate his or her own work? Does the scholar bring an appropriate breadth of evidence to his or her critique? Does the scholar use evaluation to improve the quality of future work?Glassick, C.E., Huber, M.T., Maeroff, G.I, Scholarship Assessed: Evaluation of the Professoriate. Jossey-Bass.
122Task M: Closing Evaluation of Tutorial My grasp of SOTL from Alpha to Omega is improved.The tutorial had fresh significant perspectives of value.I gained valuable resources for future use.The tutorial was worth the time I spent working through it.SD D U A SAPlease complete the above evaluation and press the button to responses. For example, if you strongly disagree with item 1, please reply 1SD, etc. Your comments are also most earnestly solicited. Thanks very much.