Presentation on theme: "1 Improving Student Writing and Thinking through Assessment General Education and Assessment: Creating Shared Responsibility for Learning Across the Curriculum."— Presentation transcript:
1 Improving Student Writing and Thinking through Assessment General Education and Assessment: Creating Shared Responsibility for Learning Across the Curriculum Association of American Colleges and Universities (AAC&U) Network for Academic Renewal Conference Atlanta, GA February 17, 2005 Teresa L. Flateby University of South Florida
2 General Education Assessment Evolution 2000 Problem Areas -- Writing skills -- Liberal Arts Mathematics Common Outcomes -- General Education Dimensions - Contemporary Issues Intellectual Development 2003 – 2004 Planned General Education Revisions
3 General Education Assessment Evolution (continued) In Process Common for all Undergraduates -- Mathematics -- Natural Sciences -- Social and Behavioral Sciences -- Fine Arts and Humanities -- Human and Cultural Diversity within the Global Context -- English Composition
4 General Education Assessment Targeted Student Outcomes Writing Achievement Cognitive Level Attainment Intellectual Development Contemporary Issues Understanding Liberal Arts Mathematics Achievement
5 Assessment Tools / Strategies Depend on Outcomes What you intend to do with results –Determine proficiency –Use for program improvement
6 Evaluating Writing Assessment Process Initially assessed writing with a holistic method Realized writing was weaker than desired But – impossible to determine specific areas of weakness
7 To address this assessment concern… An analytic scale was introduced to assess writing and cognitive levels reached. Keep in mind assessment purposes.
8 CLAQWAs Development Developed: For use by faculty whose discipline is not English –to provide consistent assessment of students writing –to encourage student feedback To be flexible –identify skills appropriate for the assignment –provide different weighting for skills To attend to cognitive levels when developing assignments and to assess levels reached.
9 Cognitive Level and Quality of Writing Assessment (CLAQWA) Scale
10 Cognitive Scale
11 Based on Madaus, 1973. Analysis Evaluation/Synthesis Application Comprehension Knowledge Branching of Blooms Taxonomy of Educational Objectives: Cognitive Domain
12 Importance of Cognitive Level Targeted in Assignments Writing suffers when unprepared Clear expectations for students and instructors / administrators Better understanding of results Why?
13 First Assignment: USF's General Education Advisory Committee and the Undergraduate Council are interested in the elements you value in the classroom. These Committees are composed of university faculty, the USF Assessment Office, and undergraduate students. In a 2-3 page essay, describe a course that would represent the ideal learning experience for you. Please be as specific and concrete as possible about what this course would include; use as much detail as you think is necessary to present clearly this ideal situation. For example, you might want to discuss what the content or subject matter would be, what the teacher would be like, your responsibilities as a student, the evaluation process that would be used, and so on. Please explain why you feel the specific course aspects you discuss are ideal for you. Essay Assignment: FYC – ENC 1101
14 Essay Assignment: Capstone Course The University is interested in the elements you value in the classroom. To address these, you have been given the opportunity to assist with developing and teaching a university course. Directions Design the ideal (discipline) college level course including one or more of these dimensions: values/ethics, ethnicity, gender, environmental issues, international and global perspectives.
15 In a well-written, typed 2-4 page essay, please describe your course and provide a title. Explain why the elements you have focused on are important. The General Education Advisory Committee and the Undergraduate Council appreciate your ideas and will consider them in their assessment of the universitys General Education curriculum. Essay Assignment: Capstone Course (continued)
16 Your essay should address the following questions: 1.What subject matter and method(s) of instruction will you include in your course? 2.What type of learning will occur? (Examples: learning facts, analyzing concepts, applying skills or ideas to new situations, creating solutions to complex problems) 3.What would you do as the teacher and what would you expect your students to do in return? 4.What types of interactions will occur among members of the class? 5.What do you expect the students to get out of the course and how would you know (assess) what was accomplished? Essay Assignment: Capstone Course (continued)
17 Verbs to Use for Constructing Prompts to Elicit Specific Cognitive Levels in Student Writing This list suggest verbs intended to guide the development of the prompt. Using these words in writing prompts will help communicate the expected cognitive level to be achieved by students. Level 4: Analysis, Synthesis, Evaluation discriminateplanweigh inferorganizeevaluate comparegeneratecombine contrastappraiseconclude createcritiquesupport designjudge Level 3: Application determineuse an approachapply a principle chartdevelopsolve a problem implementchoose an appropriaterelate prepareproceduredemonstrate Level 2: Comprehension describeestimateillustrate generalizeclassifygive an example of paraphraseexplainstate in your own words summarizepredicttranslate Level 1: Knowledge definelistname identifymatchreport labelrecallselect staterecite
18 Level 5: Transitional words, phrases, sentences and paragraphs (coherence devices) smoothly connect the papers elements, ideas and/or details allowing the reader to follow the writers points effortlessly. Level 4: Coherence devices are rarely missing and do not impact the readers understanding. Level 3: Coherence devices are appropriate when present, but effort is required to connect the writers points. Level 2: Coherence devices are attempted, but are ineffective. Level 1: Coherence devices are absent or inappropriate. Trait: Coherence Devices
19 Level 5: Details develop the main idea and provide supporting statements, evidence, or examples to explain or persuade effectively. Level 4: Details support the purpose and main idea of the paper with adequate clarity, depth and accuracy. Level 3: Details are related to the purpose and main idea of the paper but do not provide sufficient depth or clarity or accuracy to explain or persuade effectively. Level 2: Details are loosely related to the purpose of the paper but are lacking clarity, depth or accuracy. Level 1: Details do not support the purpose or main idea of the paper. Trait: Quality of Details
20 Level 5: The paper clearly has and maintains a main idea throughout. Level 4: The main idea is clear, although a rare occasional extemporaneous element is introduced. Level 3: The paper has a main idea, but additional unrelated ideas distract the reader. Level 2: The main idea is not maintained or it is unclear. Level 1: The paper lacks a main idea and appears to reflect the writers stream of consciousness. Trait: Main Idea
21 Assessment Participants Assessment Activity:Participants: General Education AssessmentFaculty, Administrators, Students Advisory Committee CLAQWA Writing Scoring TeamFaculty, TAs (undergraduate & graduate) MID Scoring TeamFaculty Cognitive Level Scoring TeamFaculty Individual Student ParticipantsHonors College Students (Essay & Questionnaire Content Analysis)
22 Analytic Writing Skills Ranked Sentence construction varies appropriately. Point of view is consistent. A main idea is present and maintained. Ideas are comprehensible. All assignment requirements are fulfilled. Appropriate audience(s) are consistently addressed. Purpose is clear and specific. Details are sufficient in quantity to develop main idea. Word choice is accurate. Grammar and mechanics. Opening supports the main idea. Coherence devices are present and appropriate. Details are sufficient in quality to develop main idea. Paragraphs support unity. Reasoning supports the main idea. Closing supports the main idea. (Lowest skills related to reasoning.)
23 Cognitive Level Distribution
24 Emphasis on Critical Thinking -- Push for smaller classes -- Funds for TAs and undergraduate assistants Focus on Process Writing -- Emphasis on feedback and revision -- Expansion of writing lab to center -- Changes in composition program Center on Learning/Student outcomes rather than faculty content centered General Education Curricular Changes Resulting from Assessment Data
25 Discoveries Value in an institution assessing writing rather than an agency (ETS/ACT) Benefits of analytic scoring over holistic scoring Efficiency of essay assignments –Qualitative and quantitative data –Writing, cognitive levels, intellectual development –Data regarding general education dimensions –Areas of strength/weaknesses in curriculum –Conversations about quality writing
26 Discoveries (continued) Students degree of pedagogical savvy Writing skill and intellectual development moderately correlated Both faculty and administrative support necessary for assessment to be successful Faculty development process http://isis.fastmail.usf.edu/ugrads/eandthttp://isis.fastmail.usf.edu/ugrads/eandt/ http://usfweb2.usf.edu/ugrads/writingsurvey/