1. June 2002QPRC 2002, Tempe, Arizona A Workshop on Assessing to the Baldrige Criteria Cheryl L. Jennings, Motorola Lynn Kelley, Textron

2. June 2002QPRC 2002, Tempe, Arizona Objective of this Workshop Begin to develop the skills needed to –Assess a Baldrige-type application to the Performance Excellence Criteria requirements –Provide feedback relevant to an organizations Key Factors Understand how to –Analyze business results in a Baldrige-type application –Report results to demonstrate performance and improvement, in any situation

3. June 2002QPRC 2002, Tempe, Arizona Outline Performance Excellence Criteria The Application Evaluation Process –The Scorebook –Writing Feedback Comments Evaluating Results Items Team Exercise Report Out Q&A

4. June 2002QPRC 2002, Tempe, Arizona The Criteria for Performance Excellence

5. June 2002QPRC 2002, Tempe, Arizona Performance Excellence An integrated approach to organizational performance management that results in customers –Delivery of ever-improving value to customers, contributing to marketplace success; organizational –Improvement of overall organizational effectiveness and capabilities; and learning –Organizational and personal learning (Criteria, p. 1)

6. June 2002QPRC 2002, Tempe, Arizona Criteria Framework: A Systems Perspective (Criteria, p. 5)

7. June 2002QPRC 2002, Tempe, Arizona Categories/Items and Point Values (Criteria, p. 9)

8. June 2002QPRC 2002, Tempe, Arizona Item Format (Criteria, p. 47)

9. June 2002QPRC 2002, Tempe, Arizona The Application Evaluation Process

10. June 2002QPRC 2002, Tempe, Arizona Baldrige Award Process (Feedback, p. 2) No Receive applications Stage 1 Independent Review Stage 2 Consensus Review Stage 3 Site Visit Review Judges select for Consensus Review? Feedback report to applicant Judges select for Site Visit Review? Feedback report to applicant Stage 4 Judges recommend Award recipients to NIST Director/DOC Feedback report to applicant

11. June 2002QPRC 2002, Tempe, Arizona Independent ReviewScorebook Key Factors (KFs) –Concise summary of most important aspects of organization environmentsOrganizational and Competitive environments relationshipsKey working relationships challengesKey strategic challenges Key Themes –Overall summary of key points in application strengthsMost important strengths identified opportunitiesMost significant opportunities (OFIs) identified ResultsMost significant strengths and/or OFIs in Results Item Worksheets comments –Single, complete comments linked to Criteria and KFs

12. June 2002QPRC 2002, Tempe, Arizona Key Factors Synthesized by Examiner from the Organizational Profile in the application –Organizational Description Environment and Relationships –Organizational Challenges Competitive Environment, Strategic Challenges, Performance Improvement System Used by Examiners to understand the organization and what is considered important by the organization

13. June 2002QPRC 2002, Tempe, Arizona Developing Worksheet Comments Criteria requirementsRead the Criteria requirements for the Item applicants responseRead the applicants response against the Criteria Key FactorsIdentify the Key Factors relevant to the Item single, complete thoughtsDevelop and write 6–10 commentssingle, complete thoughtslinked to the Criteria and Key Factors Designate each comment as a Strength (+) or an Opportunity for Improvement (OFI) (–) An OFI should be written for any Area that is not addressed

14. June 2002QPRC 2002, Tempe, Arizona Item 1.1 Organizational Leadership Basic Item Requirement –Describe how senior leaders guide your organization, including how they review organizational performance Key Factors may include –Type of organization (public, private,industry) –Number of locations or sites –Mission, Vision, Values, other cultural context –Customers and Markets –Key suppliers, partners, vendors

15. June 2002QPRC 2002, Tempe, Arizona Item 3.2 Customer Relationships & Satisfaction Basic Item Requirement –Describe how your organization builds relationships to acquire, satisfy, and retain customers, … Key Factors may include –Customer and Market segments –Main products and services delivered, and their delivery processes –Mission, Vision, Values

16. June 2002QPRC 2002, Tempe, Arizona Examples of Comments Item 1.1b (+) The Leadership Team uses a dashboard of performance measures, e.g., Customer Satisfaction, Customer Complaints, and Market Share, to review how well the organization is operating and to identify priorities for improvement and opportunities for innovation. Item 3.2a (–) Although the applicant offers access to its customers via telephone, it is not apparent that it has an approach in place for determining whether its key access mechanisms are sufficient for current or potential customers who are seeking information or who wish to make complaints.

17. June 2002QPRC 2002, Tempe, Arizona Evaluating Results Items

18. June 2002QPRC 2002, Tempe, Arizona Results Results document how the applicant measures its performance –Over time, –Against competitors performance, and –Against relevant comparative data Essentials to reporting Results Key Factors –Clearly relate to the organizations Key Factors current levels –Reveal current levels trends –Show trends comparisons –Contain appropriate comparisons

19. June 2002QPRC 2002, Tempe, Arizona Writing Comments on Results Address the following questions –Is trend positive or negative? What is desirable direction? Are explanations provided for significant changes? –Are all important results presented? Are there any gaps in the data? –Is the amount of data provided sufficient? –Are data appropriately segmented? –Do data represent both short- and long-term priorities? –Are comparative data presented, and are they appropriate? –Are data normalized? (Guidance, p. 11)

20. June 2002QPRC 2002, Tempe, Arizona Item 7.2 Financial & Market Results Basic Item Requirement –Summarize your organizations key financial and marketplace performance results by market segments, as appropriate. Include appropriate comparative data. Key Factors may include –Number and type of competitors –Key strategic challenges –Long-term strategic objectives, such as growth

21. June 2002QPRC 2002, Tempe, Arizona Examples of Results Comments Item 7.2 (+) Average Assets (Figure 7.2-4) have grown more than ten times since 1995, indicating the applicants strategy of Branch Growth (Figure 7.2-5) is putting it on track to achieve its strategic objective of $3 billion in assets by 2004. Item 7.2 (–) The applicant does not segment its financial results by customer segments, products and services, or business areas. From the results provided, the applicant may not be able to determine how well it is progressing in its key strategy of increasing Internet-based business … (Guidance, p. 11)

22. June 2002QPRC 2002, Tempe, Arizona Team Exercise

23. June 2002QPRC 2002, Tempe, Arizona TriView National Bank Case Study Prepared for use in the 2001 MBNQA Examiner preparation course A sample application written for a fictitious financial services institution applying for the Baldrige award

24. June 2002QPRC 2002, Tempe, Arizona Exercise Instructions You will be grouped into teams Allotted time: 45 minutes Identify a Timekeeper, Leader, Recorder, and Reporter Prepare –Review the relevant Criteria Identify key Criteria requirements –Read the excerpts from the application Note the results reported –Read the already-completed Key Factors Worksheet

25. June 2002QPRC 2002, Tempe, Arizona Instructions contd Evaluate application to Criteria and prepare the Item Worksheet –Select 4-6 most important Key Factors for the Item* –Write one Strength comment* –Write one OFI comment* –Record the * on flip charts

26. June 2002QPRC 2002, Tempe, Arizona Instructions contd Report Out –Key Factors –One strength comment –One OFI comment –Observations?

27. June 2002QPRC 2002, Tempe, Arizona Questions?

28. June 2002QPRC 2002, Tempe, Arizona References 2001 Criteria for Performance Excellence. Baldrige National Quality Program (BNQP), National Institute of Standards and Technology, Gaithersburg (NIST), MD. 2001 Guidance for Producing a Well-Written Scorebook. BNQP, NIST, Gaithersburg, MD. 2001 TriView National Bank Case Study Packet, including Scorebook and Feedback Report. Prepared by BNQP. Available from the American Society for Quality, Milwaukee, WI.

29. June 2002QPRC 2002, Tempe, Arizona Handout at Beginning From 2001 Business Criteria for Performance Excellence –Criteria for Item 7.1, Category and Item Description for Item 7.1, Scoring Guidelines, Comment Guidelines From TriView Application –Item 7.1 Response From TriView Scorebook –Key Factors Worksheet From Blank Scorebook –Item 7.1 Worksheet

30. June 2002QPRC 2002, Tempe, Arizona Handout after Report-out From TriView Scorebook –Recommended Scoring Range for Item 7.1 is 15–25 –Key Themes, c –Item 7.1 Customer-Focused Results

31. June 2002QPRC 2002, Tempe, Arizona Contact Info Cheryl Jennings can be reached at C.Jennings@Motorola.com Lynn Kelley can be reached at LKelley@TFSA.Textron.com