Presentation is loading. Please wait.

Presentation is loading. Please wait.

Professor Timothy Davis School of Law Wake Forest University WORKSHOP ON DRAFTING EFFECTIVE MULTIPLE-CHOICE QUESTIONS SCHOOL OF LAW SOUTHERN UNIVERSITY.

Similar presentations


Presentation on theme: "Professor Timothy Davis School of Law Wake Forest University WORKSHOP ON DRAFTING EFFECTIVE MULTIPLE-CHOICE QUESTIONS SCHOOL OF LAW SOUTHERN UNIVERSITY."— Presentation transcript:

1 Professor Timothy Davis School of Law Wake Forest University WORKSHOP ON DRAFTING EFFECTIVE MULTIPLE-CHOICE QUESTIONS SCHOOL OF LAW SOUTHERN UNIVERSITY MARCH 15,

2  Workshop Goals9:00 – 9:10  The Value of MCQs 9:10 – 9:20  The Components of MCQs9:20 – 9:30  Drafting Effective MCQs9:30 – 12:00  The Factual Vignette9:30 – 10:15  Drafting Exercises  BREAK10: :30  Drafting Effective MCQs  The Lead-in10:30 – 10:45  The Options10:45 – 11:30  Drafting Exercises  Drafting Exercise11: :00 WORKSHOP SCHEDULE 2

3 WORKSHOP GOALS  Better appreciate the value of effective MCQs  Identify the components of MCQ and the functions of each  Recognize common drafting errors  Follow guidelines for drafting quality MCQs  What are your goals?  What would you like to obtain from this workshop? 3

4  Effective tools to assess students’:  Knowledge of legal doctrine  Analytical skills (apply legal reasoning)  Ability to apply lawyering strategy  Improve students’ performance on objective exams (e.g., MPRE, MBE)  Content Coverage—assess knowledge on a broad number of topics  Grading consistency and ease  Reliable means of assessing students’ analytical abilities  Correlation between scores on essay exams and MCQs THE VALUE OF WELL-DRAFTED MCQS 4

5 Each of the following is critical to drafting effective MCQs:  The Factual Scenario/Vignette (Stem)  The Lead-In (Question)  Options (Key and Distractors) COMPONENTS OF A MCQ 5

6 Tips for Drafting Effective Vignettes  Determine the skills you want your questions to assess  Do you want your students to: analyze what’s important and what isn’t? identify a rule or an exception? spot issues?  Draft clear and precise vignettes  Helps to avoid inadvertently making a distractor a correct answer  Use common descriptors  E.g., the buyer, the seller, the florist, the man, the woman THE FACTUAL VIGNETTE 6

7 Tips for Drafting Effective Vignettes  Draft vignettes that are understandable  Students should not be required to sort through overly complex, long and confusing problems  Avoid desire to make factual vignette more engaging or realistic; this can detract from your objective  Multiple issues can be tested in a simple factual vignette  Difficulty is a function of the sophistication and plausibility of the options FACTUAL VIGNETTE 7

8 Tips for Drafting Effective Vignettes  Content is important  Develop facts that focus on important concepts; avoid testing trivial facts/obscure concepts  A well-drafted vignette sets up the legal issues that pertain to what is being assessed/tested FACTUAL VIGNETTE 8

9 Original Robert Weiss, a Pennsylvania resident, worked at a steel mill in Allentown, PA. He became ill and his doctor informed him that he needed a heart transplant. Weiss's employer provided medical insurance for him under a contract with Aetna Insurance, a Delaware corporation with its principal place of business in Hartford, CT. Weiss applied to Aetna for precertification that the policy covered the cost of the heart transplant. Aetna denied coverage on the ground that the transplant was an experimental procedure that fell outside the scope of the insurance plan. Three months later, Weiss died. Weiss's widow filed a complaint in a Pennsylvania state court alleging that Aetna violated the Pennsylvania Unfair Insurance Practices Act. The complaint demanded compensatory damages of $500,000 and punitive damages of $5 million. Aetna responded by filing a removal petition in a Pennsylvania federal district court. It argued in its petition that the case presented a federal question because federal insurance regulations preempt Pennsylvania law. Weiss's widow has moved to remand the case to the state court. Her motion should be: A. Denied, because the case raises a federal question. B. Granted, because the case does not raise a federal question. C. Denied, because the parties are diverse and more than $75,000 in damages is claimed. D. Granted, because the federal district court lacks the power to invalidate a Pennsylvania law. Key B Word Count: 227 Revised A man, who was a Pennsylvania resident, needed a heart transplant. The man’s employer provided medical insurance through an insurer, a Delaware corporation with its principal place of business in Hartford, CT. The insurer denied the man’s application for precertification of the transplant, stating the transplant was not a covered procedure. The man died three months later. The man’s widow filed a complaint in a Pennsylvania state court alleging that the insurer’s denial violated a Pennsylvania insurance statute. The complaint demanded $500,000 compensatory and $5 million punitive damages. The insurer filed a removal petition in a Pennsylvania federal district court. Its petition stated that the case presented a federal question because federal insurance regulations preempt Pennsylvania law. The man’s widow has moved to remand the case to the state court. Should the widow’s motion be granted? A. No, because the case raises a federal question. B. No, because the parties are diverse and more than $75,000 in damages is claimed. C. Yes, because the case does not raise a federal question. D. Yes, because the federal district court lacks the power to invalidate a Pennsylvania law. New key is C Word Count: 184 ILLUSTRATION 1—REDRAFTING A FACTUAL VIGNETTE 9

10 Please review this vignette. After your review, please collaborate with a partner and share your observations. We will then revise the vignette together. Mary Crawley, the eldest daughter of an old and wealthy Baton Rouge family, is the owner of a large commercial office building in downtown Baton Rouge, Louisiana called Downton Plaza. On January 15, 2006, Mary entered into a lease of Downton Plaza to Tom Branson, the owner of a company that refurbishes old-turn-of-the-century antique cars. The lease agreement included a legal description of the property and provided for rent in the amount of $4,000/month, due on the last day of each month of the term of the lease. The lease further provided for a term of five years beginning on January 15, 2006 and ending on January 15, 2011 and stated that “either party herein bound may terminate this Lease Contract at any time upon giving the other five (5) days’ written notice.” The years passed and Mary received her rents on time, and by all accounts Tom was pleased with his lease of Downton Plaza. However, when January 15, 2011 arrived, Tom did not vacate the property and, instead, remained in place. Tom intended to call Mary and discuss the potential for renewing the lease for another five (5) years, but had been so busy working on Baton Rouge’s numerous turn-of-the-century antique cars, that he completely forgot to call her. On February 7, 2011 Mary sent a letter to Tom demanding that he and his business vacate Downton Plaza “immediately.” Which of the following statements is true? (a)Tom Branson must vacate Downton Plaza by February 12, 2011, but no earlier. (b)Tom Branson must vacate Downton Plaza by February 15, 2011, but no earlier. (c)Tom Branson must vacate Downton Plaza by March 15, 2011, but no earlier. (d)Tom Branson must vacate Downton Plaza by February 15, 2016, but no earlier. (e)Tom Branson must vacate Downton Plaza immediately. ILLUSTRATION 2—REDRAFTING A FACTUAL VIGNETTE 10

11 Mary Crawley, the eldest daughter of an old and wealthy Baton Rouge family, is the owner of a large commercial office building in downtown Baton Rouge, Louisiana called Downton Plaza. On January 15, 2006, Mary entered into a lease of Downton Plaza to Tom Branson, the owner of a company that refurbishes old-turn-of-the-century antique cars. The lease agreement included a legal description of the property and provided for rent in the amount of $4,000/month, due on the last day of each month of the term of the lease. The lease further provided for a term of five years beginning on January 15, 2006 and ending on January 15, 2011 and stated that “either party herein bound may terminate this Lease Contract at any time upon giving the other five (5) days’ written notice.” The years passed and Mary received her rents on time, and by all accounts Tom was pleased with his lease of Downton Plaza. However, when January 15, 2011 arrived, Tom did not vacate the property and, instead, remained in place. Tom intended to call Mary and discuss the potential for renewing the lease for another five (5) years, but had been so busy working on Baton Rouge’s numerous turn-of-the-century antique cars, that he completely forgot to call her. On February 7, 2011 Mary sent a letter to Tom demanding that he and his business vacate Downton Plaza “immediately.” ILLUSTRATION 2---REDRAFTING A FACTUAL VIGNETTE 11

12 Break (Finally -- Please return) MCQ WORKSHOP 12

13  Effective lead-ins will:  Frame the student’s task  Pose a clear/specific question  Stated so that each option is responsive to it  Allow the student to answer the question without having seen the options THE LEAD-IN 13

14  Practices to Avoid:  Sentence fragments: E.g., “The buyer is:”  A question makes it clearer what the student is expected to answer.  Negative framing--negatively stated questions increase student anxiety and can be confusing.  E.g., Which of the following theories will be least helpful to the owner?  Difficulty is a function of the construction and plausibility of the options and not a lack of clarity. THE LEAD-IN 14

15 Predictive Outcome Lead-ins: Assess students’ ability to synthesize law and facts to predict an outcome  Examples  Is the seller likely to prevail?  Is the ordinance constitutional?  Does the man have a cognizable tort claim?  Who owns what interest in the land?  What amount of damages will the court award to the buyer?  Options associated with predictive-outcome lead-ins are often phrased in “No, because…./Yes because…. THE LEAD-IN 15

16 Dispositive Issue/Formulating Strategy Lead-ins: Focus is on the hurdle a party must overcome or standard that a court must apply to resolve a dispute  Examples  Which of the following issues will be dispositive?  What is the appropriate burden of persuasion?  What test will the judge apply?  What is the biggest obstacle to relief?  Which issue will be most critical to liability?  Which of the following standards of proof should be applied by the court?  Options associated with this lead-in are often framed in “Whether/if” statements  E.g., “Whether the evidence is sufficient to prove....” THE LEAD-IN 16

17 Engaging Lead-ins: These focus on students taking a practical approach to engaging problems. They inquire as to what a lawyer or judge should/would do in a given situation.  Examples  How should the lawyer advise the client?  What issue must the lawyer resolve before advising the client?  What information does the lawyer need to proceed?  What is the most likely explanation for the court’s decision?  Options are often stated as advice, motions, objections or instructions. THE LEAD-IN 17

18 Tips for Drafting Effective Options  Options should be concise  Options should be parallel where possible  Options should be responsive to the lead-in  Options should be listed in logical order  Distractors should be plausible in order to reduce guessing  Distractors should draw on common student misconceptions to enhance their plausibility  Difficulty is a function of the construction and plausibility of the options. OPTIONS 18

19  Practices to Avoid  Long complicated options  Vague terms (e.g., “rarely,” “usually”)  Absolute terms (e.g., “always,” “never”): options using such terms are rarely the key and savvy test takers will understand this  K-type options (e.g., “(A) I and II, (B) II and II, but not IV”): under the stress of the exam students will become tangled up in the extra analysis required to answer  Using “none of the above” or “all of the above”  May be appropriate but use with caution  Inconsistent grammar: students tend not to select options in which the grammar isn’t consistent  Humorous distractors  Non-plausible distractors and obviously incorrect distractors (eliminates options and increases students’ chances of guessing the correct answer)  Unnecessary duplication of words OPTIONS 19

20 Original The State of Wake was concerned with the quality of elementary education in both its public and private schools. It passed a statute requiring that all teachers, both public and private, be college graduates and achieve a certain score on the National Teacher Exam. If challenged by a sectarian school that insists that the primary qualification of its teachers is religious affiliation and that the state's guidelines would force it to hire some teachers from outside its denomination, the U. S. Supreme Court would: (A)Declare the statute constitutional since the state has a legitimate interest. (B)Declare the statute constitutional since the state has an important interest. (C)Declare the statute unconstitutional since the state has no legitimate interest. (D)Declare the statute unconstitutional since the state is effecting an establishment of religion. Key (A) Word Count 134 Revised A state legislature, concerned with the quality of public and private school elementary education, passed a statute requiring all public and private school teachers to achieve a minimum score on a national teacher’s exam. A sectarian school challenged the constitutionality of the statute. It argued that the qualification of its teachers is religious affiliation and the statute would force it to hire teachers from outside its denomination. Will the U. S. Supreme Court declare the statute constitutional or unconstitutional? (A)Constitutional, because the state has a legitimate interest. (B)Constitutional, because the state has an important interest. (C)Unconstitutional, because the state has no legitimate interest. (D)Unconstitutional, because the state is effecting an establishment of religion. Key (A) Word Count: 111 ILLUSTRATION 3: REDRAFTING A LEAD-IN AND OPTIONS 20

21 Revised Facts, Lead-in and Options A state legislature, concerned with the quality of public and private school elementary education, passed a statute requiring all public and private school teachers to achieve a minimum score on a national teacher’s exam. A sectarian school challenged the constitutionality of the statute. It argued that the qualification of its teachers is religious affiliation and the statute would force it to hire teachers from outside its denomination. Will the U. S. Supreme Court declare the statute constitutional or unconstitutional? (A)Constitutional, because the state has a legitimate interest. (B)Constitutional, because the state has an important interest. (C)Unconstitutional, because the state has no legitimate interest. (D)Unconstitutional, because the state is effecting an establishment of religion. Key A Word Count: 118 Alternative Lead-in and Options A state legislature, concerned with the quality of public and private school elementary education, passed a statute requiring all public and private school teachers to achieve a minimum score on a national teacher’s exam. A sectarian school challenged the constitutionality of the statute. It argued that the qualification of its teachers is religious affiliation and the statute would force it to hire teachers from outside its denomination. Will the U. S. Supreme Court uphold the constitutionality of the statute? (A)Yes, because the state has a legitimate interest. (B)Yes, because the state has an important interest. (C)No, because the state has no legitimate interest. (D)No, because the state is effecting an establishment of religion. Key A Word Count 118 ILLUSTRATION 3B-REDRACTING A LEAD-IN AND OPTIONS 21

22 Please review this vignette. After your review, please collaborate with a colleague and share your observations regarding the lead-in and options. We will then revise the lead-in and options together. Susan Sharp took out a personal loan for $10,000 from Midland Bank to finance the purchase of inventory for her beauty supply store. She signed a security agreement granting Midland a security interest in “all inventory”. Midland advanced the $10,000 on March 1 st, 2012 and immediately filed a financing statement in the proper place covering “all personal property”. The financing statement was properly indexed under Susan Sharp. On March 1, 2013, Susan married Bradley Barnes and changed the name on her drivers license to Susan Barnes. She also relocated from Winston Salem, North Carolina to Danville, Virginia that same day. On June 25, 2013, Betty Consumer obtained a judgment against Susan. The sheriff levied against this judgment and seized $5,000 worth of inventory on June 30, On July 3, 2013, Susan filed for bankruptcy. Midland Bank filed an amended financing statement in Raleigh, North Carolina on April 1, 2013 which changed Susan Sharp’s name to Susan Barnes. On July 2, 2013, who wins in a priority battle between Midland Bank and Betty Consumer over Susan’s inventory? a) Midland b) Susan c) Betty Consumer d) The parties have equal rights in the inventory ILLUSTRATION 4—REDRAFTING A LEAD-IN AND OPTIONS 22

23 On July 2, 2013, who wins in a priority battle between Midland Bank and Betty Consumer over Susan’s inventory? a) Midland b) Susan c) Betty Consumer d) The parties have equal rights in the inventory ILLUSTRATION 4—REDRAFTING A LEAD-IN AND OPTIONS 23

24 Please review this vignette. After your review, please collaborate with a colleague and share your observations. We will then revise the vignette together. NPK, Inc. is a Louisiana corporation engaged in the manufacture of natural fertilizers. NPK has seven directors, four of whom are also officers of NPK. The CEO, Chief Operating Officer (COO), Chief Financial Officer (CFO) and General Counsel (GC) also serve as directors. The other three directors are outside directors who do not work for NPK and who own no stock in NPK. The CEO is also chair of the board. The CEO, COO, and CFO are also the largest shareholders of NPK and together own 75% of NPK stock. The NPK Articles of Incorporation contain only the basic provisions required by law. During a properly convened board meeting on January 10, 2013, the CEO/Chair made a proposal that NPK adopt a new stock rule allowing all existing shareholders a right of first refusal for all future stock issuances. Under the proposal, every current stockholder will have the right to purchase his or her proportionate share of stock from any new issuance of stock. This proposal received a 3-2 vote. The COO, CFO, and one of the outside directors voted in favor of the proposal. The other two outside directors voted against the proposal. The CEO/Chair and the GC abstained from the vote. The CEO/Chair declared the proposal approved. He then suggested that the GC draft a new by-law to implement this new rule. The meeting was then adjourned. Which of the following statements is most accurate concerning the vote held at the January 10 board meeting: a)All directors voted properly and the vote was a binding decision of the NPK board. b)The GC had no valid reason to abstain from the vote, and therefore, the vote of the board is invalid. c)The CEO/Chair was required to appoint a special committee of shareholders to vote on this proposal. d)The vote on the proposal failed because two of the three votes in favor came from persons who had an interest in the proposal (i.e., the COO and CFO). ILLUSTRATION 5 REDRAFTING A MCQ 24

25 NPK, Inc. is a Louisiana corporation engaged in the manufacture of natural fertilizers. NPK has seven directors, four of whom are also officers of NPK. The CEO, Chief Operating Officer (COO), Chief Financial Officer (CFO) and General Counsel (GC) also serve as directors. The other three directors are outside directors who do not work for NPK and who own no stock in NPK. The CEO is also chair of the board. The CEO, COO, and CFO are also the largest shareholders of NPK and together own 75% of NPK stock. The NPK Articles of Incorporation contain only the basic provisions required by law. During a properly convened board meeting on January 10, 2013, the CEO/Chair made a proposal that NPK adopt a new stock rule allowing all existing shareholders a right of first refusal for all future stock issuances. Under the proposal, every current stockholder will have the right to purchase his or her proportionate share of stock from any new issuance of stock. This proposal received a 3-2 vote. The COO, CFO, and one of the outside directors voted in favor of the proposal. The other two outside directors voted against the proposal. The CEO/Chair and the GC abstained from the vote. The CEO/Chair declared the proposal approved. He then suggested that the GC draft a new by-law to implement this new rule. The meeting was then adjourned. ILLUSTRATION 5 REDRAFTING A MCQ 25

26 Which of the following statements is most accurate concerning the vote held at the January 10 board meeting: a)All directors voted properly and the vote was a binding decision of the NPK board. b)The GC had no valid reason to abstain from the vote, and therefore, the vote of the board is invalid. c)The CEO/Chair was required to appoint a special committee of shareholders to vote on this proposal. d)The vote on the proposal failed because two of the three votes in favor came from persons who had an interest in the proposal (i.e., the COO and CFO). ILLUSTRATION 5 REDRAFTING A MCQ 26


Download ppt "Professor Timothy Davis School of Law Wake Forest University WORKSHOP ON DRAFTING EFFECTIVE MULTIPLE-CHOICE QUESTIONS SCHOOL OF LAW SOUTHERN UNIVERSITY."

Similar presentations


Ads by Google