1. Comment Writing Exercise Return Examiner Training

2. 2 How do good comments add value? Criteria Application content Organization Profile The most valuable Comments/Key Themes

3. 3 The Structure of Well Written Comments Well written comments incorporate criteria language, ADLI/LeTCI, scoring language, and application language.

4. 4 Well Written Comment Guidelines Category 1-6 Does the comment link to the Criteria? Does it consist of 2-3 grammatically correct sentences (about 1000 characters)? Does it use examples from the application that illustrate the OFI/Strength? Does it include a so what if it is an OFI? Is the statement non-judgmental and non-prescriptive? Does the comment link to the scoring guidelines?

5. 5 Well Written Comment Guidelines Tie your comments to the basic, overall, or multiple level Criteria requirements that are most important to the applicant. Write process comments so they contain: a subject identified from the criteria, the application, or the scoring guidelines; verb(s) and requirements from the Criteria; examples from the application, and figure numbers, as appropriate. Use the evaluation factors (ADLI or LeTCI), and scoring language, to clearly articulate the areas of the applicant’s response to clarify the strength or provide insights that will help the applicant improve overall organizational effectiveness and capabilities. Draw linkages between an Item and the applicant's Organizational Profile. Ensure that the comment does not contradict other comments in the same or other Items or in the Key Themes Worksheet.

6. 6 The Style of Well Written Comments Use active voice and present tense (e.g., 'completes' rather than 'is completed'). Use a polite, professional, and positive tone. Use vocabulary/phrasing from the Criteria and the Scoring Guidelines. Describe what is missing if something "is not clear". “Not clear” comments must be resolved if the applicant has a site visit. If they don’t do it based on site experience then say they don’t do it. In lieu of the applicant’s name, use such phrases like "the applicant" or "the organization" to refer to the applicant in the independent and consensus scorebooks. Use the applicant's name in final feedback reports. Use the applicant's language/terminology when appropriate.

7. 7 “Don’ts” to avoid in comments 1.Avoid parroting 2.Avoid being prescriptive 3.Avoid going beyond the criteria

8. 8 “Do’s” for well-written comments (refer to comment guideline in workbook)

9. 9 Example Strength Comment The applicant identifies their core competencies as the participating dentist network; expert claims administration; extraordinary customer service and performance culture. A SWOT analysis is used in the annual strategic planning process to identify core competencies. Correlation analysis is performed on satisfaction survey data and is used to determine key drivers of customer satisfaction. Initiative action plans accentuate core competency strengths. Deployment is evidenced by the use of bi-annual CAG meetings, ensuring alignment with the mission and the integration of data for easy access across all departments and customer segments Red-Criteria Green-ADLI Blue-Application Pink-Key Factor

10. 10 Well Written Comments Don’t…. – Go beyond the requirements of the Criteria or assert your personal opinions. – Be prescriptive by using "could", "should", or "would". – Be judgmental by using terms such as "good", "bad", or "inadequate". – Tell the applicant that they aren’t doing enough of something – Comment on the applicant's style of writing or data presentation. – Use jargon or acronyms unless they are used by the applicant – Only “Parrot” the application or the Criteria. Provide only enough language to add clarity—seek to add value rather than restate information.

11. 11 How Could This Comment Be Improved? Application language: Share Food uses a biennial 12 step Strategic Planning Process that involves community leaders, volunteers, member agencies, and donors. Strategic challenges are identified in the “Current State” step which includes inputs from the SWOT analysis, environmental scan, and funding mandate review. The Strategic Planning Process was initiated in 1997 and restructured in 2004 as a result of benchmarking analysis of food banks and other nonprofit organizations, and feedback from the state award process. Comment: Share Food uses a biennial 12 step Strategic Planning Process (SPP) that involves community leaders, volunteers, member agencies, and donors. Strategic challenges are identified in the Define step which includes inputs from the SWOT analysis, environmental scan, and funding mandate review. This is proof that the organization has a great systematic approach to the SPP and extensive cycles of learning in 1997, and 2004.

12. 12 How Could This Comment Be Improved? Application language: The manager’s vision helps Our Store to maintain a strong focus on customer satisfaction. The manager has worked hard to instill the belief that leadership and responsibility should be present at all levels. This rewards everyone at Our Store with the responsibility of planning, carrying out, and evaluating their own work. Comment: Our Store is in the beginning stages of building an approach to setting a vision for customer satisfaction, this includes the manager working very hard to instill the beliefs in leadership and responsibility. Everyone in the store receives a reward and this will ensure that everyone will have ownership in the organization and the ability to evaluate their own work, which should improve performance overall.

13. 13 How Could This Comment Be Improved? Application language: Our Hospital builds relationships with customers through a variety of methods including patient enrollment, which serves as an orientation to enabling services; Patient-Family Advisory Boards, which have recommended and implemented improvements to patient care; and high-visibility community health activities. Using patient surveys, conducted randomly after discharge, Our Hospital determines the issues that lead to poor relationship results and uses this information to implement new relationship-building processes. The Patient-Family Advisory Board is an example of the results of this process. Comment: Customer relationships are built through a variety of methods including Patient enrollment, Patient-Family Advisory Boards, and community health activities. Surveys are conducted randomly after patients are discharged. The Patient-Family Advisory Board is used to determine issues that lead to poor relationships and implements new relationship- building processes.

14. 14 How Could This Comment Be Improved? Application language: Customer groups are identified annually as part of the Strategic Planning Process using Our Hospital’s vision/mission/values as a focus. By analyzing demographic data from multiple sources, the cross-location Service with Spirit Team (SWST) is able to identify the gaps, look at disparities, and identify potential customers. Deployment of this process through multiple cycles has resulted in the development of several services designed to meet the unmet needs of customers in the Our Hospital’s service area. Comment: Through the SPP, customer groups are annually identified using the MVV as a focus. The SWST uses demographic data to identify gaps, disparities and identify potential customers. This process has been deployed throughout the organization, and through multiple cycles of refinement the customers needs in the service area have been met.

15. 15 Well Written OFI Comment Guidelines – May be used to give credit to the applicant for some areas that HAVE been addressed “While the applicant does x, y, z, it is not evident that a, b, and c are performed.” – May begin with a statement about what is missing: “There is no evidence that…” “There is no indication of how…” “It is unclear…” “It is not apparent that there is a …” – Must tie to criteria language, (ADLI), and/or to Key Factors that the applicant has identified in the Organizational Profile – Must include a “so what” (Why does the applicant care about this? Why would they take action?) Usually these come from Key Factors / Org Profile. The closer the ‘so what’ can be to identifying the applicant’s specific needs, the better.

16. 16 Example OFI Comment Although cross functional teams are chartered and aligned with the SPP, there is no indication of processes used by the teams or otherwise by the applicant to design and innovate its work processes to meet key requirements. There is no indication of how design and innovation processes address opportunities for agility and improvements in cycle time and productivity. It is also unclear how the team work process improvement approach is evaluated for annual process changes or improvements. An annual evaluation of strategic team alignment and improvements may provide efficiency and effectiveness in the accomplishment of strategic objectives, initiatives, and action plans. Red-Criteria Green-ADLI Blue-Application Pink-Key Factor

17. 17 How Could This Comment Be Improved? Application language: Share Food uses a biennial 12 step Strategic Planning Process that involves community leaders, volunteers, member agencies, and donors. Strategic challenges are identified in the “Current State” step which includes inputs from the SWOT analysis, environmental scan, and funding mandate review. The Strategic Planning Process was initiated in 1997 and restructured in 2004 as a result of benchmarking analysis of food banks and other nonprofit organizations, and feedback from the state award process. Comment: Although Share Food uses a biennial 12 step Strategic Planning Process (SPP) that involves community leaders, volunteers, member agencies, and donors, there is no evidence of how blind spots are identified, nor how time horizons are set and how the SPP addresses those horizons. If the applicant implemented a more extensive benchmarking process that identified blind spots and included time horizons, there may be more of an opportunity for this organization to have a more successful Strategic Planning Process.

18. 18 How Could This Comment Be Improved? Application language: The manager’s vision helps Our Store to maintain a strong focus on customer satisfaction. The manager has worked hard to instill the belief that leadership and responsibility should be present at all levels. This rewards everyone at Our Store with the responsibility of planning, carrying out, and evaluating their own work. Comment: Although the manager has worked hard to instill the belief that leadership and responsibility should be present at all levels, there is no indication of a systematic approach, nor is there deployment at any level, no integration, nor are there any cycles of learning.

19. 19 How Could This Comment Be Improved? Application language: Our Hospital builds relationships with customers through a variety of methods including patient enrollment, which serves as an orientation to enabling services; Patient-Family Advisory Boards, which have recommended and implemented improvements to patient care; and high-visibility community health activities. Using patient surveys, conducted randomly after discharge, Our Hospital determines the issues that lead to poor relationship results and uses this information to implement new relationship-building processes. The Patient-Family Advisory Board is an example of the results of this process. Comment: There is no systematic approach to building customer relationships, or in conducting patient surveys. And although there is a Patient-Family Advisory Board there is no approach described for how issues are determined and how new relationship-building processes are developed.

20. 20 How Could This Comment Be Improved? Application language: Customer groups are identified annually as part of the Strategic Planning Process using Our Hospital’s vision/mission/values as a focus. By analyzing demographic data from multiple sources, the cross-location Service with Spirit Team (SWST) is able to identify the gaps, look at disparities, and identify potential customers. Deployment of this process through multiple cycles has resulted in the development of several services designed to meet the unmet needs of customers in the Our Hospital’s service area. Comment: Although customer groups are identified annually in the SPP, and the Spirit Team (SWST) identifies gaps, disparities, and identifies potential customers, there is no indication how this information is deployed throughout the organization and how the information is used to meet the unmet needs of customers in the area.