1. Nominal Group Process (NGP) A well researched technique (Delbecq et al., 1986) that is effective in facilitating a group to come to the best combined judgements about any subject in a short period of time, (typically 3-4 hours). The design of the NGP encourages the best thinking of all participants.

2. Nominal Group Process Its design reduces the influence of individuals who would tend to dominate discussions, and increases the input given by those that are shy.

3. Why Use NGP There are many ways to make clients aware of problems in their instructional system. For example, their boss may tell them a problem exists and which solution to apply. This type of solution is limited in that interest in the problem may only last as long as the administrative pressure does.

4. Why Use NPG Using a democratic means to identify problems and solutions often results in more enduring change. If people are told they need to change the way they do their job, without understanding why, they often get defensive.

5. Overall Purpose Create awareness in clients of problems in their systems. Create possible solutions to problems. Added advantage: once people agree that a problem exists, they become more receptive at looking at possible solutions.

6. Critical Steps 1) Determine the initial question to be answered. E.g., Which elements of the Instructional Design M.A. program need to be improved? 2) Identify appropriate respondents, and get agreement from them to participate. E.g., those who have completed the program, and those who have taught it.

7. Critical Steps 3) Make sure the work space is set up appropriately. 4) Set groups of six to twelve around tables. Use a horseshoe arrangement, with a facilitator and a flip chart between the open ends of the horseshoe.

8. Critical Steps 5) Present introductory statements and rules to the group. Tell them how the technique is effective when the its few rules are followed. They will have to identify and rank items relevant to their task. The process should take 3-4 hours and they will get a break in the middle.

9. The process is made up of four activities: a) generating items in response to the task, b) stating the items, c) discussing the items, d) prioritizing the items. Some parts are done individually, and others are done as a group. The job of the facilitator is to direct participants through these activities.

10. Rules The facilitator is to make sure these are followed. Participants must attend the entire session. They are not to speak to each other. They may speak only through the facilitator. They must begin and complete activities as designated. They may ask the facilitator to clarify items, but may not otherwise comment on items put forward by other participants.

11. Critical Steps 6) Hand a response sheet to everyone. Give them 5-10 minutes to generate their list of items independently. 7) Have participants begin the item- generating activity. 8) Stop when the large majority has finished writing. Tell them they can continue adding items throughout the group listing process.

12. 9) Have individuals begin the round-robin listing of the items they have generated. Ask the first individual to the left to state one of her best items. Write items on the flip chart large enough for the entire group to see. 10) Provide clarification as needed. (If you can’t provide it ask the person who wrote the item to clarify). 11) Go to the next person, and continue this process until the group runs out of items.

13. 12) Pass blank ranking forms to the participants and tell them to rank the most important items posted on the flip chart sheets. Have them select five when the total is greater than 30. Also, ask them to write the item number and rank items from 5 to 1. (5 being the most important item) 13) Record their rankings. 14) Add the accumulated ranks for each item, and circle the highest-ranked items.

14. 15) Determine the items for which there is the widest disagreement for stimulating discussion in the next round. 16) Begin the justification process. Remind participants they are to address their questions and answers only to you. Their respective responses offer new insights and information to the group that will help them reassess their ranking of items. 17) Begin the reassessment phase. Give them blank forms to reprioritize. This time they weight each item, distributing a total of 100 points.

15. 18) Add up the ranks and weights for each item, and present this information. 19) Have them agree on which of the prioritized items should be dealt with first by their system.

17. Cost-Benefit Analysis (CBA) CBA is a technique designed to assist decision makers in identifying a preferred choice among possible alternatives. The estimated quantitative and qualitative effects of proposed alternative solutions provide a relatively objective means of comparison for the decision maker(s).

18. Rationale Evidence that a proposed innovation will have a positive impact on the system is one of the strongest arguments for its adoption. Your chances of getting a particular solution adopted are strengthened by providing an unbiased cost-benefit analysis that demonstrates the value of your innovation over the alternatives.

19. Critical Steps 1) Define the environment for which an alternative solution will be chosen. A) Acquire a copy of the instructional objectives to be met. B) List relevant characteristics of the target audience, from existing records and/or questionnaires. C) List possible constraints on making a change, (e.g., budget, time, etc.) D) Establish control values for costs, time, learner outcomes, etc.

20. 2) Design the analysis. Complete these steps. A) Determine standards, based on program objectives, used to compare alternative solutions. B) Identify costs and benefits for each alternative. C) Assess listed costs and benefits. Establish true costs by estimated PERT costs with experts and/or vendors. D) Establish the scope and dimensions of the analysis. Establish if elements will be evaluated on a qualitative or quantitative basis. E) Determine the data to be collected.

21. 3) Collect the data. A) Plan the data format. Matrices such as databases and spreadsheets allow easy access and updating of data. B) Gather data from original sources when possible; through interview, questionnaire, and observation. C) Record all sources of data for eventual CBA report reference.

22. 4) Perform the analysis. Consider the following: A) Estimate the economic and quantitative values for the different alternatives. B) Include an analysis of the qualitative effects of each alternative. C) Structure the analysis to be as unbiased as possible, (e.g. have two or more team members do parallel independent analysis).

23. 5) Organize and represent the results A) Arrange results so they permit easy identification of the effects of the alternatives. B) Provide qualitative assessment. C) Present all material in a useful manner to the decision maker. D) Taylor your presentation to the special needs of your client. E) Keep the CBA report brief. F) Provide a brief report summary.