1. Guideline Development
At WHO
Courtesy of
Susan L. Norris, MD MPH MSc
Guidelines Review Committee Secretariat
10 October 2016

2. Outline What is a WHO guideline?
Guideline development processes at WHO
Principles underlying WHO guidelines
Role of Guideline Development Group
Rules of procedure: consensus

3. What is a WHO guideline?
“A WHO guideline is any document, whatever its title, that contains WHO recommendations about health interventions, whether they be clinical, public health or policy interventions.”
 ”A recommendation provides information about what policy-makers, health-care providers or patients should do. It implies a choice between different interventions that have an impact on health and that have ramifications for the use of resources.”
"Guidelines are recommendations intended to assist providers and recipients of health care and other stakeholders to make informed decisions. Recommendations may relate to clinical interventions, public health activities, or government policies." WHO 2003, 2007
Any document containing recommendations about health interventions (treatment,diagnosis, policy)
Name of document not relevant
Health information products that are not guidelines are:
Standards for manufacturing health technologies or standard operating procedures for organisations
`how to` documents – eg how to implement a service, how to set up a research project

4. Types of guidelines at WHO
Standard
Full systematic review and guideline development process
6 months to 2 years
Compiled or consolidated
Include GRC-approved recommendations
Rapid advice guidelines
Compressed and abbreviated process, potential for bias
1 to 3 months
Interim
Standard or rapid advice guideline processes; Often narrow scope
Anticipate short shelf-life: follow with standard guideline

5. Oxman et al, Lancet 2007;369:1883-9

6. Oxman et al. Lancet 2007
“Findings: Systematic review and concise summaries of findings are rarely used for developing recommendations. Instead processes usually rely heavily on experts in a particular specialty, rather than representatives of those who will have to live with the recommendations or on experts in particular methodological areas.”

7. WHO Guidelines Review Committee
Established in 2007 to develop and implement procedures to ensure that WHO guidelines are:
consistent with internationally accepted best practices
appropriately based on evidence.
transparent
Members from headquarters and all 6 regions
5 external members
Decisions are made by consensus

9. Principles for WHO Guidelines
Well-defined scope and target audience
Broad and representative guideline development group
Disclosure and management of all secondary interests (COI)
A priori development of key question for systematic reviews
Systematic and comprehensive evidence retrieval, synthesis
Quality assessment of the body of evidence for each question
Formulation of recommendations based on the evidence and other explicit considerations.
Adherence to WHO reporting standards
Usable document: relevant, applicable, user-friendly
Include a plan for implementation and updating

10. FINAL GUIDELINES: Top reasons for non-approval
Evidence to recommendations
Inadequate explanation for determination of strength of recommendations
Disclosures of interest and funder
DOI not presented in sufficient detail
Document presentation
Executive summary, copy editing, technical editing, organization, layout
Within these categories, some of the major issues were…

11. Guideline development at WHO1
Scope the guideline 2
Set up GDG and External Review Group 3
DOI and manage conflicts of interest 4
Formulate questions (PICO) and
Choose relevant outcomes
GRC approval of guideline development proposal 5
Evidence retrieval, assessment, synthesis
(systematic review(s))
GRADE - evidence profile(s) 6
Formulate recommendations: GRADE
Include explicit consideration of:
Benefits and harms
Values and preferences
Resource use
GRC approval of final guideline 7
Disseminate, implement (a 8
Evaluate impact 9
Plan for updating 11

12. WHO Guidelines…
Must meet the highest quality standards for evidence-based guidelines
Must be based on high-quality systematic reviews of all relevant evidence
Use GRADE, which provides an explicit approach to:
Assessing the quality of the evidence across studies and outcomes
Translating evidence to recommendations
Incorporate multiple processes to minimize bias and optimize usability
Must incorporate transparency in all judgments and decision making processes
Updating: when

13. Knowledge gaps
Identifying gaps in the evidence is an important aspect of systematic reviews and guideline development
Guideline Development Groups are in an ideal position to define an agenda for future research
Especially important when evidence is sparse
Want to mention one other important issue:
Although we will not have time to formally work on and prioritize a research agenda, we should all keep this in mind over the next 2 days, as future research will be an important section of the final guideline document.

14. Rules of Procedure: Group decision making
WHO recommendations should be based on consensus
Defined as general agreement among the decision makers
Minor disagreements can be addressed in the Remarks Section of the guideline
Co-chairs to work with the Secretariat to find solutions
If consensus cannot be reached, voting
Simple majority, anonymous or hand-raising at Chair’s discretion (observers and secretariat do not vote)
Tied vote within the GDG: 2 co-chairs to identify a solution or break the tie by consensus between 2 of them
Voting is discouraged, consensus is strongly preferred

15. Rules of Procedure: Group decision making
Every effort should be made to try
to achieve consensus!

16. Role of Guideline Development Group
provide input into the scope of the guideline;
assist the steering group in developing the key questions in PICO format;
choose and rank priority outcomes that will guide the evidence reviews;
examine the GRADE evidence profiles or other assessments of the quality of the evidence used to inform the recommendations and provide input;
interpret the evidence and formulate recommendations taking into account benefits, harms, values and preferences, feasibility, equity, acceptability, resource requirements and other factors, as appropriate; and
review and approve the final guideline document before submission to the GRC.

17. Composition of Guideline Development Group
relevant technical experts;
end-users, such as programme managers and health professionals, who will adopt, adapt, and implement the guideline
representatives of groups most affected by the recommendations in the guideline, such as service users and representatives of disadvantaged groups;
experts in assessing evidence and developing guidelines informed by evidence;
other technical experts as required.
Complete DOI; COI assessed and managed by WHO staff
Participate as individuals; do not represent institutions

18. Role of meeting attendees
Meeting Observers
Technical consultants: provide information, opinions, guidance, provide comments only upon request
Do not participate in deliberations on the recommendations
No DOI needed
May represent an institution or group
WHO Steering Group and other WHO staff
Prepare initial planning proposal
Support development of recommendations by the GDG
Draft the guideline document
Are subject to WHO Staff Regulations for COI and confidentiality

19. Role of meeting attendees
Guideline methodologist
Helps to develop recommendations informed by evidence using a transparent and explicit process
Provides technical expertise on systematic review and guideline development methods
Completes DOI; generally has no COI
Other technical experts: Systematic Review Team
Presents the review; provides technical expertise on review methods

20. Rules of Procedure: Confidentiality
CONFIDENTIALITY UNDERTAKING
 The World Health Organization (WHO), acting through its Department of ………………, has access to certain information relating to ……………., which information WHO considers to be proprietary to itself or to parties collaborating with it (hereinafter referred to as "the Information").
All meeting attendees who are external to WHO should have signed this agreement: observers, consultants, GDG members

21. Questions? ?