doc.: IEEE /042r2 Submission January 2003 Mary DuVal TI and Ian Gifford, ConsultantSlide 1 Project: IEEE Working Group for Wireless Personal Area Networks (WPANs) Submission Title: [TG3a Down Selection Process Discussion] Date Submitted: [14Jan03] Source: [Mary DuVal] Company [Texas Instruments] Address [P.O. Box , MS 847 Plano, TX USA 75086] Voice:[+1 (972) ], FAX: [+1 (972) ], Source: [Ian Gifford] Company [Consultant] Address [23 Kelshill Road, Chelmsford, MA USA 01863] Voice:[ ], FAX: [ ], Re: [02/385r0, -02/470r3] Abstract:[TG3a Call For Proposal timelines and processes contribution; updated.] Purpose:[The purpose of this submission is to update the TG3a thinking on the CFP, Evaluation, Scoring, and Voting Processes that kicked off during the Nov02 Plenary SG discussion topic on the possible CFP and Down Selection Processes. The next steps are to have the TG agree to a single process and a timeline.] Notice:This document has been prepared to assist the IEEE It is offered as a basis for discussion and is not binding on the contributing individual(s) or organization(s). The material in this document is subject to change in form and content after further study. The contributor(s) reserve(s) the right to add, amend or withdraw material contained herein. Release:The contributor acknowledges and accepts that this contribution becomes the property of IEEE and may be made publicly available by

doc.: IEEE /042r2 Submission January 2003 Mary DuVal TI and Ian Gifford, ConsultantSlide 2 IEEE Working Group for Wireless Personal Area Networks (WPANs) TG3a Down Selection Subcommitee (SC) Down Selection Process Discussion

doc.: IEEE /042r2 Submission January 2003 Mary DuVal TI and Ian Gifford, ConsultantSlide 3 Contents Down Selection Subcommittee (SC) Work To Date, unfinished business next steps Overview of the process and timelines Selection Process –Evaluation –Down Selection

doc.: IEEE /042r2 Submission January 2003 Mary DuVal TI and Ian Gifford, ConsultantSlide 4 Next Steps Committee analysis of scoring – how to organize? –Definition of committee analysis activity to be determine by end of January meeting – discussion encouraged Review Evaluation Annex text for 02/105 –Proposed text is located in 02/471r4 –Review in January Down selection Voting Procedure (02/465r1) –Members are encouraged to review steps 3 – 10 –Suggest concall to discuss 3 – 10 to identify areas of concern (12/4 and 12/11 at 11 am CST – host?) –Formal editing of this procedure will continue in the January meeting starting at step 3

doc.: IEEE /042r2 Submission January 2003 Mary DuVal TI and Ian Gifford, ConsultantSlide 5 SC Overview Nov02 to Jan03 The SG3a/TG3a CFP was released 3Dec02 The TG3a PAR was approved by NesCom/StdsBD on 11Dec02 ConCalls held on 4Dec02 and 11Dec02 and the minutes can be found in – 02/491r1

doc.: IEEE /042r2 Submission January 2003 Mary DuVal TI and Ian Gifford, ConsultantSlide 6 Session #22/FLL SC - Contributions -03/031r0 [03031r0P802-15_TG3a-PHY- Selection-Criteria.doc] –Annex A based on SC –02/471r4 -03/041r0, r1 [03041r1P802-15_TG3a-Down- Selection-Voting-Procedure.doc] –Based on -02/465r1, -02/487r0 -03/042r0 [03042r0P802-15_TG3a-Down- Selection-Process-Discussion.ppt]

doc.: IEEE /042r2 Submission January 2003 Mary DuVal TI and Ian Gifford, ConsultantSlide 7 Overview of IEEE Process

doc.: IEEE /042r2 Submission January 2003 Mary DuVal TI and Ian Gifford, ConsultantSlide 8 Call for Applications, Intentions, Proposals, and Down Selection Process Legend: CFI = Call For Interest CFA = Call For Applications CFI/CFP = Call For Intent/Call For Proposals

doc.: IEEE /042r2 Submission January 2003 Mary DuVal TI and Ian Gifford, ConsultantSlide 9 The CFI/CFP Process The figure to the right depicts the current state of the TG3a CFP Process thinking. The WG requires a CFI to prepare the agenda. Typically a CFP runs days and the CFI is 30 days BEFORE CFP deadline.

doc.: IEEE /042r2 Submission January 2003 Mary DuVal TI and Ian Gifford, ConsultantSlide 10 Down Selection Procedure, 02/491r1 After the 4Dec02 ConCall ChuckB decided to create a flow chart based on –02/465r1 (- 03/041r1) The steps in the graphic represent paragraphs in the procedure.

doc.: IEEE /042r2 Submission January 2003 Mary DuVal TI and Ian Gifford, ConsultantSlide 11 Proposed CFP Timeline Nov02 Session #21/Kauai, HI USA –We approved and then released the CFP on 3Dec02. Jan03 Session #22/Ft Lauderdale, FL USA –We are planning that the TG3a approve the docs from the SG3a. Mar03 Session #23/Dallas, TX USA –Proposals will be first heard in this Mar03 session. May03 Session #24/Singapore, Singapore –If too many proposal for the Mar03 session, this will be the overflow. –We should spend part of this session discussing proposals presented, answering technical questions/concerns (i.e. put the diehard engineers in a room and let them verbally duke it out). –Spend time on conference calls discussing proposals to let everyone get comfortable (before and after this session). Jul03 Session #25/San Francisco, CA USA –Down Selection and Voting will occur here –1st opportunity for automatic quorum after Mar03 presentations Sep03 Session #26/TBA –Potential to start the drafting process - proposals for draft improvements should be entertained at this time Nov03 Session #27/Albuquerque, NM USA –TG3a drafting process...

doc.: IEEE /042r2 Submission January 2003 Mary DuVal TI and Ian Gifford, ConsultantSlide 12 SG3a Future Planning 2003 JJASONDJFMAMJJ D JFMAM The CFA was released on 11Dec01 and closed on 21Jan02. The CFI/CFP was released on 3Dec02 – CFI closes 3Feb03 and CFP closes 3Mar03. A SON You are here CFP 2002 CFA D PAR Draft

doc.: IEEE /042r2 Submission January 2003 Mary DuVal TI and Ian Gifford, ConsultantSlide 13 Ad Hoc Summary and Current Status

doc.: IEEE /042r2 Submission January 2003 Mary DuVal TI and Ian Gifford, ConsultantSlide 14 Ad Hoc Summary The ad hoc session was called to order by Ian Gifford, at 7 p.m. No minutes were taken other than the following slides. Attendees: –The Ad Hoc Editing Team consisted of: Jim Allen, Steve March, Steve Turner, John Santhoff, Anuj Batra, Rick Roberts, Matt Welborn, Ian Gifford (facilitator), Len Miller, and Gregg Rasor. Thank you! We recessed to the Hotel Bar at 8:30 p.m.

doc.: IEEE /042r2 Submission January 2003 Mary DuVal TI and Ian Gifford, ConsultantSlide 15 Ad Hoc Status The Ad Hoc committee reviewed the TG3a minutes –03/012r1 and applied all the edits that were referred to the ad hoc Down Selection Subcommittee from the Task Group 3a. –Contributions: -03/041r2, Ad Hoc 13Jan03 edits -03/041r3, Post Ad Hoc 14Jan03 edit

doc.: IEEE /042r2 Submission January 2003 Mary DuVal TI and Ian Gifford, ConsultantSlide 16 Ad Hoc Status (cont.) Open Issues List: –CFP Presentation order in Mar03? Pick from a hat. – accept as SOP –Panel Sessions? After Initial Proposals (Step 2) - accept When 2 Proposals remain (Step 7) - accept

doc.: IEEE /042r2 Submission January 2003 Mary DuVal TI and Ian Gifford, ConsultantSlide 17 Ad Hoc Status (cont.) Open Issues List: –Voting Definition, Format, Examples Low Hurdle Vote (Step 3) – modified to Consider/Not Consider –e.g., TGg see –03/041r5 Elimination Vote (Step 7) –e.g., TG3 see –00/373r3 and –00/374r3 Roll Call Vote (Step 9) –e.g., TG3 see –00/373r3 and –00/374r3 –Jim Allen edits [03041r1P802-15_TG3a- comments - JDA.doc] Editorial - accepted

doc.: IEEE /042r2 Submission January 2003 Mary DuVal TI and Ian Gifford, ConsultantSlide 18 Ad Hoc Status (cont.) 14Jan03 Open Issues List: –RickA comments (verbal, 14Jan) - accept –Chang comments: Merged and non Merged present - accept –JohnB edits Active Proposals on Panel - accept Prep time prior to Panel – rock n roll

doc.: IEEE /042r2 Submission January 2003 Mary DuVal TI and Ian Gifford, ConsultantSlide 19 Ad Hoc Status (cont.) 15Jan03 Open Issues List: –Reviewed –02/491r1 for consistency and acceptance –BobH, JohnB, ChuckB comments –RickR and IanG prepared r5 See next slide

doc.: IEEE /042r2 Submission January 2003 Mary DuVal TI and Ian Gifford, ConsultantSlide 20 Ad Hoc Status (cont.) 15Jan03 Open Issues List: –Corrected flow for Step 2. Deleted 1 st sentence moved to intro of Step 3. –Corrected text and example in Step 3. Changed vote to ballot to be clear. Deleted “abstention counted” sentence. Corrected example text and table for consistency. –Corrected text in Step 7. Added text to address Soo-Young Chang’s comment. –Corrected text in Step 8. Added text to define deliverable to WG. –Corrected text in Step 9. Created a stop on round 2 of confirmation.

doc.: IEEE /042r2 Submission January 2003 Mary DuVal TI and Ian Gifford, ConsultantSlide 21 Backup Slides

doc.: IEEE /042r2 Submission January 2003 Mary DuVal TI and Ian Gifford, ConsultantSlide 22 Down Selection Process Options Considered with Straw Poll Results –Separate Evaluation/Down Selection Voting: 42 –Evaluation is the Down Selection Voting (combined) : 3 –Down Selection Voting only: 0 –Abstain: 14 Red = Winner straw poll

doc.: IEEE /042r2 Submission January 2003 Mary DuVal TI and Ian Gifford, ConsultantSlide 23 Evaluation Process Evaluation is Really 2 discussions (or phases) –Criteria Importance Level Mandatory/Optional ABC –A: Mandatory requirement –B: Important desired requirement –C: A nice to have requirement Weighted values (0 – 10) None –Scoring Pass/Fail Pugh Matrix –Better (+), Same, Worse (-) than a Baseline Solution Rating (n > 2) None

doc.: IEEE /042r2 Submission January 2003 Mary DuVal TI and Ian Gifford, ConsultantSlide 24 Evaluation Process Options Matrix Scoring Criteria Importance Level Mandatory/ Optional ABC RatingWeighted Values Pass/Fail  Pugh Matrix  Rating (0-5) 

doc.: IEEE /042r2 Submission January 2003 Mary DuVal TI and Ian Gifford, ConsultantSlide 25 Evaluation Process NoCriteria Importance LevelScoringStraw Poll Count 1Mandatory/OptionalPass/Fail0 2Mandatory/OptionalRating (n >2)7 3ABC RatingPass/Fail0 4ABC RatingRating (n>2)40 5Weighted ValuesPugh Matrix0 6Weighted ValuesRating (n>2)5 7NonePass/Fail 8NonePugh Matrix 9NoneRating (0-5) 10None 11Abstain1 Grey = Voted off the straw poll Red = Winner straw poll

doc.: IEEE /042r2 Submission January 2003 Mary DuVal TI and Ian Gifford, ConsultantSlide 26 Criteria Importance Level Results

doc.: IEEE /042r2 Submission January 2003 Mary DuVal TI and Ian Gifford, ConsultantSlide 27 Criteria Importance Level Results (cont.)

doc.: IEEE /042r2 Submission January 2003 Mary DuVal TI and Ian Gifford, ConsultantSlide 28 Scoring Discussion Document scoring method in Annex for inclusion in IEEE P /105 Alternate PHY Selection Criteria –Contribution in 02/271r4, text to be reviewed in January Decisions –Extent of Scoring: tabulated information (13) vs. tabulated information with committee analysis (35) vs. abstain (1) Definition of committee analysis activity to be determine by end of January meeting –Criteria to Score: only most important (A’s only) vs. all criteria (no objections) –Definition of N > 2 Rating How many levels desired: 3 levels (26) vs. 5 levels (23) vs. abstain (4) Levels labeling: + / 0 / - (24) vs. a worded version (21) vs. abstain (3) Red = Winner straw poll

doc.: IEEE /042r2 Submission January 2003 Mary DuVal TI and Ian Gifford, ConsultantSlide 29 Down Selection Voting Procedure Options Considered with Straw Poll Results –Ranking vote (lowest rank voted off): 2 –Vote for desired proposal (lowest # of votes is off): 14 –2 staged vote (eliminate low support proposals, vote for desired proposal): 22/32 –Two votes per voting member (lowest number off): 18/21 –Abstain: 5/5 Red = Winner straw poll

doc.: IEEE /042r2 Submission January 2003 Mary DuVal TI and Ian Gifford, ConsultantSlide 30 Down Selection Procedure Activity Ad hoc group met Tuesday evening to develop proposed text for sub-committee Sub-committee reached consensus on items 1 & 2 (of 10) – see 02/465r1 for current text –Procedure must be set in by the end of January meeting –Members are encouraged to review steps 3 – 10 –Suggest concall to discuss 3 – 10 to identify areas of concern (12/4 and 12/11 at 11 am CST – host?) –Formal editing of this procedure will continue in the January meeting starting at step 3