1. Katherine Perez Jacqueline Peña Office of Planning and Institutional Effectiveness January 2009

2.  SLO matrix  SLOs and Corresponding Assessment Methods  Workshop: Revising SLOs and Assessment Methods  Reporting Results and Use of Results  Workshop: Writing the Results and Use of Results  PO matrix  POs and Corresponding Assessment Methods  Reporting Results and Use of Results  Workshop: Revising POs and Assessment Methods + Writing Results and Use of Results  Additional individualized assistance

3.  SACS Accreditation  Key SACS Deadlines:  September 10, 2009—Report due  March 8-12, 2010—Onsite Review  Key IE Deadlines:  February 1, 2009—Fall 2008 SLO matrices due  June 1, 2009—Spring 2009 SLO and PO matrices due  Continuous improvement  Institutional Effectiveness  Sound, research-based and assessment-based decisions

4. Student Learning Outcome (Stated in Measurable Terms) Assessment Method Results (Data Summary and Analysis) Use of Results for Improving Student Learning Link to the Mission:

5.  Criteria:  Can be observed and measured  Relates to student learning towards the end of the program  Reflects an important concept  Formula: Who + Action Verb + What  Students will identify and apply the key components of a learning system.  Students will identify and apply the five key components of a learning system.

6.  Principle 3.4.12  The institution’s use of technology enhances student learning and is appropriate for meeting the objectives of its programs. Students have access to and training in the use of technology. (Technology use)  Required for all undergraduate programs  Optional for 2007-2008 SLO matrices  Required for 2008-2009 SLO matrices

7.  General (not appropriate for SACS):  Students will use information technology to gather and disseminate information.  More Specific:  Graduates of the program will demonstrate the ability to use appropriate computer technology, software, and the Internet to complete their capstone research papers.  Much more Specific (appropriate for SACS):  Students will be able to effectively demonstrate information technology skills by locating and retrieving information on economic topics and issues, published research in Economics and related fields, and/or by finding information about the generation, construction, and meaning of economic data for their final research projects.  Students will write and present a capstone project that requires the use of Word, Excel, PowerPoint, and information technology.

8. Action Verbs[Who + Action Verb + What] KnowledgeComprehensionApplicationAnalysisSynthesisEvaluation ListSummarizeApplyAnalyzeCombineAssess DefineDescribeDemonstrateSeparateIntegrateRank DescribeInterpretIllustrateExplainModifyTest IdentifyPredictSolveConnectCreateMeasure ShowDistinguishExamineClassifyDesignJudge ExamineEstimateModifyArrangeComposeCritique StateParaphraseClassifyComparePrepareDiscriminate FindCompareCalculateContrastWriteSupport LabelContrastComputeInferFormulateHypothesize InferConstructOrderConstruct Implement Note: This table reflects the original Bloom’s Taxonomy Table. The modified version places evaluation as the fifth column and synthesis (creating) as the sixth column.

9. 1. Artifact 2. Collection of the artifact  Where, when, and how  Census versus sampling 3. Criteria  Minimum standards on a rubric/scale or the percentage of correct items 4. Evaluation of the artifact  Faculty panel or external evaluators (reliability)  Rubrics, embedded questions, etc.

10. Common Direct MeasuresCommon Indirect Measures Standardized exams Exit examinations Portfolios Pre-tests and post-tests Locally developed exams Papers Oral presentations Behavioral observations Thesis/dissertation Surveys or questionnaires Student perception Alumni perception Employer perception Focus groups Interviews Student records

11. Student Learning Outcome (Stated in Measurable Terms) Assessment Method Results (Data Summary and Analysis) Doctoral students will identify, define, and apply key literary theories (including formalism, historicism, mimesis, post-colonialism, structuralism, post- structuralism, psychoanalysis, modernism, and postmodernism) to their specialization areas in the field of English literature. After completing all their courses, English literature doctoral students will take the theory comprehensive exam, which will be divided into five key literary theories (one theory per exam question) that the students learn throughout their coursework. All comprehensive exams will be evaluated by a faculty panel of 3 members using a department rubric. For each of the five literary theories, students will achieve a 3 or better on a 4-point rubric: 1 = poor 2 = acceptable 3 = good 4 = excellent Use of Results for Improving Student Learning

12.  Summary of Results  Direct measures  Test items  Performance as determined by rubrics  Indirect measures  Surveys and questionnaires  Interview and focus group data  Format  Narrative  Tables or charts  Analysis of results  If pertinent, explain results in a narrative form by interpreting results or using qualitative analysis of the data.  Every student learning outcome must have at least:  One set of results  One student learning improvement strategy (use of results)

13.  All the students passed the assessment.  75% of the students met the criteria for success.  Our students passed the dissertation defense on the first attempt.  75% of the students (n=15) achieved a 3 or better on all the rubric categories for the capstone course research paper.

14. Frequency of Student Results for all Four Categories of the Research Paper (n=20) 1 NOVICE 2 APPRENTICE 3 PRACTITIONER 4 EXPERT PERCENTAGE MEETING CRITERIA Grammar2 (10%)0 (10%)8 (40%) 80% (n=16) met the criteria. Essay Structure 4 (20%)0 (15%)11 (55%)2 (10%)65% (n=13) met the criteria. Coherence of Argument 2 (10%)0 (35%)10 (50%)1 (5%)55% (n=11) met the criteria. Application of MLA Rules 3 (15%)0 (25%)12 (60%)0 (0%)60% (n=12) met the criteria. Criteria: Students will achieve a 3 or better on a 4-point rubric on all five sections of the master’s level thesis.

15. Student Learning Outcome (Stated in Measurable Terms) Assessment Method Results (Data Summary and Analysis) Doctoral students will identify, define, and apply key literary theories (including formalism, historicism, mimesis, post-colonialism, structuralism, post- structuralism, psychoanalysis, modernism, and postmodernism) to their specialization areas in the field of English literature. After completing all their courses, English literature doctoral students will take the theory comprehensive exam, which will be divided into five key literary theories (one theory per exam question) that the students learn throughout their coursework. All comprehensive exams will be evaluated by a faculty panel of 3 members using a department rubric. For each of the five literary theories, students will achieve a 3 or better on a 4-point rubric: 1 = poor 2 = acceptable 3 = good 4 = excellent 20 students took the theory comprehensive exam during the 2008-2009 academic year. 70% of the students achieved the minimum criterion (i.e. scoring a 3 or better on all five theory components). 20% of the students scored less than 3 on at least 1 theory component. 10% of the students (n=2) scored less than a 3 on all five theory components and had to be rescheduled to retake the test during the summer during the summer B session. Use of Results for Improving Student Learning

16. DODON’T DO focus on making specific improvements based on faculty consensus. DON’T focus on simply planning for improvements or making improvements without faculty feedback. DO address specific program improvements that will impact student learning. DON’T address assessment improvement plans such as revising the rubric. DO use concrete ideas. DON’T write vague ideas. DO state strategies that are sustainable and feasible. DON’T use strategies that are impossible to complete in one year considering your resources. DO use strategies that can improve the curriculum and help students learn outside of courses. DON’T focus on only improving the curriculum.

17.  Target met. Will continue to monitor.  The faculty will meet and revise the three introductory courses.  A larger sample will be obtained.  We will revise the rubric and have a calibration session with the faculty prior to evaluating the student papers each semester.  A capstone course will be created that emphasizes research and thesis writing methods.  A student resource center will be created with computer equipment, writing tutors, and statisticians who can assist our students with course- and thesis-related writing and research.

18. Student Learning Outcome (Stated in Measurable Terms) Assessment Method Results (Data Summary and Analysis) Doctoral students will identify, define, and apply key literary theories (including formalism, historicism, mimesis, post-colonialism, structuralism, post- structuralism, psychoanalysis, modernism, and postmodernism) to their specialization areas in the field of English literature. After completing all their courses, English literature doctoral students will take the theory comprehensive exam, which will be divided into five key literary theories (one theory per exam question) that the students learn throughout their coursework. All comprehensive exams will be evaluated by a faculty panel of 3 members using a department rubric. For each of the five literary theories, students will achieve a 3 or better on a 4-point rubric: 1 = poor 2 = acceptable 3 = good 4 = excellent 20 students took the theory comprehensive exam during the 2008-2009 academic year. 70% of the students achieved the minimum criterion (i.e. scoring a 3 or better on all five theory components). 20% of the students scored less than 3 on at least 1 theory component. 10% of the students (n=2) scored less than a 3 on all five theory components and had to be rescheduled to retake the test during the summer during the summer B session. Use of Results for Improving Student Learning 1.The department will assist faculty in organizing student-led comprehensive exam study groups a year prior to the expected examination deadline. 2.A theory unit will be added to each literature course so that students are constantly exposed to multiple literary theories throughout their coursework. 3.In addition, the department will create and distribute a study list that includes major topics covered on the comprehensive exams, useful websites for tutorials and reviews, and a bibliography of texts that are critical for success with the five theoretical concepts.

19. Student Learning Outcome (Stated in Measurable Terms) Assessment Methods Results (Data Summary and Analysis) Undergraduate English students will communicate effectively in written format. During their final semester, in their senior seminar course (PPP 4677), students will write a 10-page research paper on a critical topic as applied to an English, Caribbean, or American novel. 20% of the students’ papers will be randomly selected and evaluated by a faculty panel of 3 members using a rubric that measures: (1) Grammar, (2) Essay Structure, (3) Coherence of Argument, (4) Application of MLA rules. Students will achieve a 3 or better on a 4-point rubric: 1 = novice 2 = apprentice 3 = practitioner 4 = expert 100 students took the seminar in 07-08. 20 papers were sampled. 100% met the minimum criterion for success (i.e. scoring a 3 or better on all four components). Use of Results for Improving Student Learning 1.Undergraduate English students will be required to visit the Center for Academic Excellence once each semester to work with a writing tutoring for at least one 60-minute session. 2.Undergraduate English students who have difficulty with grammar, essay structure, coherence of argument, or application of MLA rules at any point during their academic program will be referred to a writing tutor at the Center for Academic Excellence. 3.Every undergraduate English course will require at least one 5-page paper that requires the use of MLA rules and principles. 1234 G00128 E0010 C00155 M00200 71%29%

20.  Overview and Comparisons:  Give an overview or summary of all the outcomes together  Discuss trends that you have seen over the years  Explanations  Provide qualitative explanations for poor results or exceptionally high results  Notes and documentation affecting results  Response rate (e.g. Only 50% of the students completed the project.)  Inter-rater reliability (e.g. 2 faculty members reviewed the artifacts and the inter-rater reliability was only 60%.)  Assessment Commentary  Revise or create instrument (e.g. artifact, rubric)  Modify assessment methods (e.g. data collection, sampling, criteria, evaluation process)

21.  Write or revise your Results and Use of Results.  Results:  Clearly state the results of each PO assessment (table or narrative).  Use of Results:  Explicitly state how you will use the results to improve the program during the following assessment cycle/year.  Summary Page: 1. Overview of results and use of results (looking at the big picture) 2. Trends over the last year or several years 3. Comparison to national or state trends or data 4. Explanations for poor or unexpected results 5. Notes or documentation that could explain the results 6. Assessment improvement plans

22. END OF THE SLO PORTION What Is Next?  10-minute overview of Program Outcomes  PO writing assistance  Continued SLO writing assistance

23.  Program-level outcomes  Focus on student success (not student learning)  Formula = Who + Action Verb + What  Examples: Graduates seeking employment in the field will find such employment within 6 months of graduation. Graduates will be satisfied with advising and mentoring services. Faculty will publish at least one journal article or book chapter every two years.

24. Program Outcome (Stated in Measurable Terms) Assessment Method Results (Data Summary and Analysis) Full-time students will graduate from the English doctoral program within 7 years of program admission. The program administrative assistant will keep track of students’ progress in the program from date of admission, making updates for each student at the end of each academic semester. 10 students graduated from the English doctoral program during the 2008-2009 academic year. The program assistant analyzed the completion timeframe for each individual student and discovered that all of the graduating students spent over half of their graduate time working on the dissertation (3-6 years). Use of Results for Improving Student Learning 1.Doctoral advisors and dissertation chairs will be required to meet with their students twice a semester to go over the program of study, the students’ progress, and the students future goals and timelines. 2.The college will support the graduate student organization in developing dissertation work/study groups, for all incoming students in the program. The dissertation groups should meet once a month to share drafts, establish new goals, create completion timelines, and do any other activities that the group members find helpful for successful completion of the program. 3.Students will be mailed and emailed warning letters after 4 years in the program reminding them that they need to graduate within 7 years of program admission. Time of Completion for the 10 Students 5 students6 years 2 students7 years 2 students8 years 1 student9 years

25.  Take 15 minutes to review or write at least one program outcome with corresponding assessment methods, results, and use of results.  PO Formula: Who + Action Verb + What  Assessment Methods Formula: 1.State the instrument/method/tool (e.g. survey, national exam results). 2.Explain the data collection steps (e.g. when, where, how). 3.State the success criteria (minimum standards for success).  Results:  Clearly state the results of each PO assessment (table or narrative).  Use of Results:  Explicitly state how you will use the results to improve the program during the following assessment cycle/year.

26.  Marta Perez  Director  perezma@fiu.edu  305-348-2733  Maria Corrales  Coordinator  corrales@fiu.edu  305-348-0459  Katherine Perez  Coordinator  katherine.perez@fiu.edu  305-348-1418  Jacqueline Peña  Coordinator  jpena@fiu.edu.  305-348-1367  Mayelin Felipe  Computer Specialist  mfelip01@fiu.edu  305-348-0115  Karla Felipe  Computer Specialist  kgarcia@fiu.edu  305-348-0115  Amanda Berhaupt  Graduate Assistant  aberh001@fiu.edu  305-348-2731  Randhir Kaur  Graduate Assistant  rkaur001@fiu.edu  305-348-2731