Presentation on theme: "Masterclass in Writing RfPB Grant Applications"— Presentation transcript:
1 Masterclass in Writing RfPB Grant Applications Carole FoggSophie HyndmanHelen Lloyd28th January 2011Presented January 2011; updated and adapted for web, January 2012
2 Welcome! Introductions Aim and Objectives of the Day 2
3 Who are we and what can we do for you? RDS South Central: Research Design Support for NHS and Social Care ResearchersWho are we and what can we do for you?
4 Some of our A-Team: Multi-disciplinary team: Statisticians Clinical TrialistsEpidemiologistsQualitative ResearchersHealth EconomistsPatient/Public ExpertsHealth Psychologists
5 What can the RDS SC do to help you? For all aspects of grant preparation, advice on:Sources of fundingLiterature search and reviewDeveloping and refining research objectivesStudy design, including health economics & statisticsPlanning teams and budgetsPatient and Public Involvement (PPI)Research Advisor, with access to specific experts as requiredResearch Design Advisory PanelsPotential for collaborationCommunications: , newsletter, websiteTraining: research master classes
6 Question: In what way is a research grant application like a general election campaign?
7 They both require the right balance between Answer:They both require the right balance between
8 AimTo support researchers in developing successful research grant proposals, specifically, Research for Patient Benefit applications
9 Objectives To provide: An overview of funding applications, and how they relate to research governance and ethics applicationsAn overview of NIHR funding schemes and the purpose and scope of RfPBAn understanding of what reviewers expect to see in an RfPB grant applicationSome guidance on identifying what makes ‘good’ and ‘bad’ grant applicationsAn introduction to the structure and content of the RfPB formAn understanding of the importance of patient and public involvement in RfPB researchAn understanding of methodological issues important to reviewersAn understanding of the resources and time required to undertake a research projectAn introduction to other sources of research support
10 Part 1 Research in the NHS: Funding Applications in Context
11 Important initiatives Good Clinical Practice (GCP) and the UK Medicines for Human Use (Clinical Trials) Regulations 2004Responsibilities and accountabilities for clinical trials of investigational medicinal productsResearch Governance Framework for Health and Social Care (2005)Informed by above, but for research more generallyBest Research for Best Health (2006)Infrastructure and systemsSalvatore Vuono.
12 The purpose of these initiatives Research GovernanceTo protect the safety, dignity, rights and wellbeing of research participantsTo promote useful, ethical, valid, safe and affordable researchTo prevent futile, unethical, invalid, dangerous and extravagant researchTo provide research standardsTo define responsibilities and accountabilitiesTo set out delivery and monitoring systems to ensure standardsBest Research for Best HealthTo make the UK a recognized centre of healthcare research excellence by providing a good research infrastructureTo build research capacityTo fund research to improve health and social careTo improve and manage knowledge and information systems to simplify research processes and maximise usefulness of research
13 The result of these initiatives Among other things…A range of funding schemes for health researchA system of research approvals to ensure that research is well regulated and ethical
14 What are the main ‘hoops’ to jump through? All research carried out in the NHS requires:A sponsorEthics approvalR&D approvalsMost research will require:FundingSome research will requireOther approvals (e.g. Clinical Trial Authorisation)Honorary contracts or research passports1 ) Hula hoop moment from Circus Smirkus performance provided courtesy of Wikipedia. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2
16 Steps to take1.... 2. Find a sponsor 3. Apply for research funding 1. Write proposal/ protocol4. Apply for ethics approval*7. If required, attend to research passports/ honorary contracts6. If required, attend to other approvals** IRAS5. Apply for R+D approval*1. May not always be in this order!Image from Office Online
17 Steps (1) Write a research protocol and (3) Apply for funding Q. What is the difference between a research protocol and a research grant application?A research protocol is the definitive document that will inform the process of the research. Some R&D offices will insist on a GCP approved protocol, and provide templates for this (regardless of whether the project is a Clinical Trial of an Investigational Medical Product (CTIMP) or not)The research protocol can be used as the basis for applying for ethics and funding, and there will be an overlap, but these latter documents will have different emphases, so it is not simply a matter of cutting and pastingSeeDownloads 16 & 17 for GCP approved templatesImage from Office Online
18 Step (2) Find a sponsor i.e.: Q. Why do you need a sponsor?A. You need an organisation to be accountable for the researchi.e.:Responsibility for the initiation and overall management, including quality assurance (research protocol, team, research environment)Responsibility for the finances associated with the studyEnsure all approvals obtained, e.g. ethics, clinical trial authorisationArrangements for handling of any Investigational Medicinal Products (IMP)Ensure Good Clinical Practice (GCP), monitoring and reporting (including safety reporting)Image from Office Online
19 Q. Who can be a sponsor?Usually:Chief Investigator’s employing institution, e.g. a university or NHS trustA funding organisationWhere research is for training purposes, the research supervisor (on behalf of their employer)Private sector e.g. pharmaceutical industry
20 Step (4) National Research Ethics Committee (NRES) approval http://www Role:Protect rights, safety, dignity and well-being of research participantsFacilitate and promote ethical research of potential benefit to participants, science and society. Advice on: information sheets and consent forms, GCP, procedures, use of personal data etc.Apply: online via Integrated Research Application System (IRAS)https://www.myresearchproject.org.uk/Image from Office Online
21 Step (5) Local trust research and development office (R&D) approval Role:To give permission to conduct research in the siteIf sponsor, to undertake that role; if hosting research (sponsored by others), to undertake audit of a proportion of studiesTo ensure all relevant research-related contracts are in placeAdvice available:Variable and dependent on personnel/ time: NRES applications, financial aspects of study from trust perspective, more general research adviceImage from Office Online
22 Directly to R&D Office for all non portfolio research. When do you meet them?Variable and dependent on personnel/ time: for high risk/ drug related/ device related studies, as soon as you have a protocol; some offices may be interested in these and other studies at the development stage.Apply:Via IRAS Coordinated System for gaining NHS Permission (CSP) for research qualifying for the NIHR Clinical Research Network PortfolioDirectly to R&D Office for all non portfolio research.
23 Step (6) Other approvals IRAS captures the information needed by:Administration of Radioactive Substances Advisory Committee (ARSAC)Gene Therapy Advisory Committee (GTAC)Medicines and Healthcare products Regulatory Agency (MHRA)Ministry of Justice (National Offender Management Service) HSC research officesNRES / NHS / HSC Research Ethics Committees National Information Governance Board for Health and Social Care (NIGB).Social Care Research Ethics CommitteeImage from Office Online
24 Step (7) Research Passports and Honorary Contracts Q. What is a research passport?A set of signatures and checks from your employing organisation required prior to issuing an honorary contract or letter of access.Q. What is an honorary contract?A contract with an NHS trust for a researcher whose employer is not that NHS trust.What is a letter of access?A letter permitting a researcher whose employer is not that NHS trust into the trustImage from Office Online
25 Always check with trusts what level of access they consider you need Q. When are honorary contracts needed?When: (a) the research is hosted in the NHS and (b) the researcher has no contract with the NHS and (c) the researcher’s activities in the NHS site will involve interacting with individual patients in a way that has a direct bearing on the quality of their care.Q. When are letters of access needed?When (a) and (b) above are present, but not (c)Always check with trusts what level of access they consider you need
26 Why are honorary contracts needed? Patients in receipt of research procedures come under the NHS duty of care and the NHS indemnity scheme applies to the researchers and their activities.The researcher is accountable to the NHS organisation for their activities, explicit supervision arrangements are in place, and the researcher is made aware of the relevant NHS policies with which they must comply in their activities.The researcher’s substantive employer understands the research activities and has undertaken all the necessary checks in relation to the researcher’s suitability to carry out those activities.Where do I apply to get an honorary contract?A. Usually Human Resources Department of NHS Trust where you wish to do the work.
27 (1) Involving a Comprehensive Local Research Network (CLRN) Steps you MIGHT have to take:Can work with investigators to:Provide NHS Support Costs for research projectsFacilitate permissionsHelp with recruitment of sites and patientsHelp with data collectionGeneral advice on the research process in their area of expertise.Image from Office Online
28 (2) Involving a Clinical Trials Unit (CTU) Can work with investigators to:Develop new trials (design, collaboration, funding, communication with Clinical Research Network, regulatory and governance issues)Manage funded trials (questionnaire design, set-up, recruitment, pharmacovigilance, data management, monitoring, analyses, report preparation)Image from Office Online
29 Part 2 – Research for Patient Benefit (RfPB) & The Importance of Sc pe Image courtesy of Wikimedia commons (Public Domain)
30 NIHR Research Programmes Research for Patient Benefit (RfPB)Programme Grants for Applied Research (PGfAR)Invention for Innovation (i4i)Health Technology Assessment (HTA)Efficacy and Mechanism Evaluation (EME)Public Health Research (PHR)Health Services and Delivery Research (NIHR HS&DR)- formerly 2 programmes: Health Services Research (HSR) & Service Delivery and Organisation (SDO)
31 Research for Patient Benefit offers… The opportunity to conduct research that:arises from daily practice in the NHSis developed between NHS and other partnersis open to topic and methodshows tangible health and health care benefits for patientsamplifies the patient voicethat encourages collaborationcloses the loop between research and practice – will be implemented or drive change
32 Aims / sc peTo support projects in health services and public health researchQualitative, quantitative or mixed method research that will:study the way NHS services are provided and usedevaluate whether interventions are effective and provide value for moneyexamine whether alternative means for providing healthcare would be more effective
33 Aims / sc pe RfPB will also fund research that will: pilot or assess the feasibility of projects requiring major applications to other funding bodiesformally assess innovations and developments in healthcareProposals which have emerged from interaction with patients/service users and public are particularly welcomeSystematic reviews may be funded as part of a larger project or as stand alone studies. For more details, see the RfPB FAQ on systematic reviews.
34 OUT of sc pe:Lab-based or basic science research, including research based on animalsSetting up or maintaining research unitsProposals which are solely service developmentsProposals which are solely audit, surveys, needs assessment, technology development (although may be included as part of an integrated research study)Images from Office Online
35 Read the guidance...Each RfPB competition has associated Guidance for Applicants that covers:aimsscopeeligibility etcFor general enquirescallOrOr visitImage from Office Online
36 Selection Criteria and Eligibility Applications judged on:the quality of the research proposedthe significance and potential benefit of the research to the NHSAll NHS organisations and other providers of NHS services in England can apply (For joint NHS/university applications funding is awarded to the NHS partner)Probably with academic partnersPotentially with strong component of service user involvement.Applications with collaborations in social care and third sector providers of health and social care are also encouragedIf strong case, can include academic partner outside England.Image from Office Online
37 The Application Process Full application deadlinePreliminary scrutiny and is it in scope?External peer and lay reviewFull committee assessmentFinal ratificationNotification of outcomeImages from Office Online
38 Overall, a wide range of RfPB projects funded Based in a diversity of settings and care groupsConcerned with clinical and non-clinical topicsUsing a diversity of methods and research techniquesSee the detailed report on the first three competitions –Prioritising Patients
40 A grant application, like a picture, should be: ClearUnambiguousEasy on the eye ☺Easy on the brainPhoto by Remy Steinegger; Copyright : World Economic Forum. Licensed under the Creative Commons Attribution-Share Alike 2.0 Generic license (http://creativecommons.org/licenses/by-sa/2.0/deed.en)Source: Wikipedia
41 It should not be :Jumbled upUnclearDifficult to decipherConfusing
42 Or indeed something you didn’t intend it to be Original Author: David Angell (adapted by Odder). Licensed under Creative Commons Attribution 2.0 Generic Licence (http://creativecommons.org/licenses/by-sa/2.0/deed.en). Source: Wikipedia
43 Part 3 RfPB APPLICATIONS (a) What would you expect to see in an RfPB grant application?
44 Part 3 RfPB APPLICATIONS (b) Examples of ‘good’ and ‘bad’ grant application sections Group/interactive workFor examples of good & bad grant application sections, see [weblink]Simon Howden
45 Part 3 RfPB APPLICATIONS (c) What a reviewer expects to see in an RfPB application form
46 Overall style Limit use of overly technical language, buzzwords and acronyms Tech Speak (defined by Edward Tenner) “Post colloquial discourse modulation protocol for user status enhancement. It's a referential system for functional-structural, microscopically specific macroscopic-object redesignation. It's a universal semantic transformation procedure. It's a holophrastic technocratic sociolect. It's a meta-semiotic mode for task specific nomenclature." Get the idea?Buzzword An important-sounding word or phrase connected with a specialized field or group that is used primarily to impress laypersons: “‘Sensitivity’ is the buzzword in the beauty industry this fall.”Alphabet soup Language laced with acronyms. "An RFP was issued by DOE; ORNL responded for the ORR."Reference:‘Tech Speak’ as defined by Edward Tenner (1986) Tech Speak, Random House, in above website.
47 Research detailsLead Applicant DetailsLead Applicant CVJoint Lead Applicant(s)Co-applicant(s) detailsResearch & Development OfficeHistory of the ApplicationPatient and public involvementCase For Support – Part 1Case for Support – Part 211. Case For Support – Part 312. Management and Governance13. Intellectual Property and Innovation14. NIHR Infrastructure15. Finances16. Monitoring information17. Suggested reviewers18. Supporting Documentation19. Declarations and signatures
48 (1) Research details Title Host organisation Region Duration Proposed start date(Application type) – Pilot, Feasibility, or Main Study*(Theme Call Name) – Only fill this in if you are applying for a specific theme call, e.g. dementia*For definitions, see:
49 Research Title Descriptive, yet concise Contains keywords relevant to the researchStudy question and main outcome measure to be includedIf a trial, also include intervention and populationState if pilot/ feasibilityClear from title that study is in scope – e.g. for RfPB, has short-medium term patient benefit
50 Title: What can go wrong Whilst you may think some titles are genuinely funny:Parasites as weapons of mouse destruction.Journal Animal Ecology Mar;77(2):201-4.They may be viewed by some as inappropriate:Targeting promiscuous signaling pathways in cancer: another notch in the bedpostTrends in Molecular Medicine Dec;10 (12):
51 Title: What can go wrong And some can appear very strange indeed:Who Moved My Cheese by Dr Spencer MorganPutnam Adult (1998)A bestseller!Image from Office Online
52 (2-5) Applicants Lead applicant Current Research Commitments and how this application will fit within current research activitiesLead applicant’s C.V.Joint lead, Co-applicants. N.B. There is now a system for co-applicants to formally confirm involvement, add C.V.s, and approve the final application. The latter must be done before submission.Consider:Kudos associated with being leadNot necessarily a big-wig but need proper support if juniorTends to be person initiating the researchContract duration (especially universities), i.e. if your contract expires before project, can you be lead?Q. What is the difference between ‘co-applicants’ and ‘research team’?A. A research team may consist of more people than simply the applicants, but N.B. the application will be judged primarily by those appearing on the form
53 (6) Research and Development Office Host must be an NHS organisationHost R&D office contact detailsImage from Office Online
54 (7) History of This Application If previous RfPB, NIHR may well look to see what the initial feedback was and how the current application has changed, BUT always viewed as a NEW application (no resubmission policy)If previously submitted elsewhere, just for their informationDetails of related applicationsDetails of concurrent applications are not asked for, but RfPB would obviously want to know if a submitted project gets funded from another source
55 (8) Patient and Public Involvement Image courtesy of healthcaregovernancereview.wordpress.com
56 What is Patient and Public Involvement?An active partnership between the patient/public and researchers, NOT the use of patients/public as ‘subjects’ to be studied (e.g. to be interviewed or assessed) by researchersFor example:Working together with patients to develop a good research proposalCommenting on the methods or processes of theresearch, being actively involved in the research processImage from Office Online
57 More power in the decision making process of the research Examples of PPIBeing part of the research teamHelping design the researchBeing a member of project reference groupWriting lay summaries and patient information sheetsCarrying out questionnaires with the participantsMore power in the decision making process of the researchImage from Office Online
58 Nature of PPI in Research Consultation:Asking people who use your services for their views on your research to help inform decision makingE.g. Commenting on the ability to understand lay summariesCollaboration:Ongoing partnership with members of the public throughout the course of the research project.E.g. A member of public on the project steering group.User-Led/ User Controlled:Members of the public lead the research and are in control of the research. This is often, through a community or voluntary organisationE.g. Service users may decide on the research question; or manage the research itself
59 Proper PPI in Research IS NOT: Asking the patient or the public to answer research questions and provide data, i.e. asking them to be a ‘guinea pig’ in your research – enrolling them into the studyListening to patients to design your service better – to elicit service design suggestionsAsking for feedback about a web based intervention – to evaluate an interventionXImage from Office Online
60 Why Involvement?To prevent researchers making mistakes that are obvious to patientsTo guide financesFor greater transparency, making research with patients,for patients, that is more relevant to patients.Obligation to the tax payerTo meet the requirements of the NHS ConstitutionTo meets the requirements of funders
61 Can you spot Involvement? The following are all extracts taken from National Institute of Health Research grant proposalsWhich are Involvement and which are not?Image from Office Online
62 "We plan a steering group for the project with four patient representatives as equal members of the research team."InvolvementParticipationNone of theseUnsure
63 "Individual service users who have been referred "Individual service users who have been referred ... via this and other routes will be interviewed regarding their experiences."InvolvementParticipationNone of theseUnsure
64 "Our patients agreed with us that the study would deliver important information and that the research was timely and important."InvolvementParticipationNone of theseUnsure
65 What can the RDS Offer?We provide a lay review service as part of our pre-submission review panel and as a separate activityWe can advertise your project through our PPI network and help to identify patients to work with youWe can provide financial support to patients or members of the public working on projects supported by the RDS up until the point of submission.We can provide PPI advice from our staff on the PPI team.
66 David Castillo Dominici What can the RDS do?Give guidance on payment structures.Provide assistance in writing lay summaries usually by getting lay members to write them either with you or for you.We have useful guidance helping you consider what involvement to use to get the best out of involvement in your project.David Castillo Dominici
67 How can we work together? Maybe you are part of, or know of, a group that wants to get involved in research, if so register the group via the RDS websiteResearchers can advertise their projects through our systemsIf you would like to get involved individually as a patient or a member of the public register with us via the RDS websiteImage from Office Online
68 What reviewers expect to see: Patient & Public Involvement (PPI) Have the needs of the service users been considered?Have you made any assumptions about service users or the co-operation of other involved parties which may not actually be met?Are there lay co-applicants?Are there lay people on the project steering group?Have the relevant national / local organisations been contacted for their input? Are there plans to involve them in the study design / study management / in dissemination of the results?Is there evidence of consulting the INVOLVE guidelines on PPI?State if PPI input at study design stage was not undertaken and say why this was the case (N.B. all efforts must be made...)Is the PPI element ‘tokenistic’, or has real thought and planning been put into it?
69 (9) Case for Support - Part 1 Aims and objectivesPlain English (Lay) and Scientific SummariesBackground and rationaleWhy research is needed now
70 Aims and ObjectivesNeed absolute clarity of objectives / primary outcome / secondary outcomes, and ensure that these relate to RfPB scope and are not contradicting the titleIs this definitive / exploratory / preliminary work? What would the upshot be if it is successful in the case of the latter?If there is a null hypothesis, is it clear and appropriate?Do any qualitative aspects of the research have clearly stated objectives / questions / intentions?If a pilot or feasibility study, make this clear, and also how it would lead on to full trial
71 Q. What is the difference between a feasibility study and a pilot study? Feasibility Studies “Can this study be done?” Used to estimate important parameters needed to design the main study, e.g. standard deviation of the outcome measure; willingness of participants to be randomised and clinicians to recruit them; number of eligible patients; characteristics / design of proposed outcome measure; follow-up rates, response rates, adherence/compliance ratesPilot studies are a version of the main study that is run in miniature to test whether the components of the main study can all work together, e.g. to test recruitment, randomisation, treatment, and follow-up assessments(http://www.netscc.ac.uk/glossary/#glos6)
72 Abstract in plain English (Lay Summary) Concise, convincing summary of:Problem (e.g. disease burden)Research question and rationaleDesign and methodExpected outcomesExpected impactBUT absolutely understandable to the lay person:No clinical or research jargonNo difficult wordsInvolve a PPI representative!
73 (Scientific) Summary Concise, convincing summary of: Problem (e.g. disease burden)Research question and rationaleDesign and methodExpected outcomesExpected impact A ‘Mini-Me’ of the main protocol!
74 Background / Rationale This section needs toPACK a PUNCH!!!Using the literature:Define the research issueJustify WHY the research is necessary – clear case for clinical importance and impact - E.g. Disease / condition burden (local / national figures)Back up logic of aims and objectivesProvide evidence of best design approachHighlight if study fits into a desired but under-represented area in the funding stream - E.g. social care, minority groupsImage from Wikimedia Commons (Public Domain).
75 Relevance of proposal to RfPB Look at RfPB scope in GuidanceThink of the:Likely benefits to patientsImplications for health servicesPotential impact on policy / service deliveryMake sure you are showing how your work fits in withthe RfPB remitImage from Office Online
76 Why is this research needed now? What evidence is available to justify the study at this time: make a caseIs the research addressing the right questions in the subject area at the right time in relation to, for example, the availability of knowledge, or the development of technology, Government priorities?Are you presenting a convincing argument?Don’t forget this is a competitive process!
77 (10) Case for Support Part 2: Research Plan & Methodology Image: NASA
78 You should include:Study design – appropriate to meet aims and objectivesSetting (location, population)Sample – sampling frame, eligibility criteriaSample sizeIntervention (if relevant)Details of randomisation if RCTAppropriate data types and sourcesOutcomes/ other measures – validated/appropriate for the populationProceduresData analysis methodsConsider:How will potential sources of bias be avoided/ taken care of? Who is blinded and how will this be achieved?FeasibilityBear in mind what will be reported (e.g. CONSORT, STARD etc)
79 Image by Xedos4, freedigital photos.net Study populationClearly defined and consistent with study title/objectivesEligibility criteria – how generalisable are the results to the NHS??gender?disease severity?comorbidities?location?new diagnosis?previous treatment?concomitant medications?exposure?ageImage by Xedos4, freedigital photos.net
80 Intervention (i) Clear description of intervention ?product / technology/therapy ?timing?dose ?length of exposure?number of sessions/treatments?staff administering treatment (training, quality assurance, experience, preferences)Allocation - randomisation method – blocks / stratification / matchingBlinding – who, how and why?Balance between standardisation and tailoring to the patientWill it be affected by patient/ carer adherence?
81 Intervention (ii) Why was it chosen? Evidence of piloting / testing or feasibility/ effectiveness from the literatureClear description and evidence-base for control / comparison group(s)For Example,Why placebo / alternative treatment chosen?Why 3 weeks treatment rather than 6?
82 Intervention (iii) Feasibility and other benefits Provision for gathering information on the patients / staff view of the research, whether it works or notPatient benefit of interventionRecoveryEmpowermentIndependenceIncreased confidence in their outlookBenefits added on existing standard of care
83 OutcomesMeasurements must accurately reflect the impact of the interventionIf surrogate / proxy measures are used, give evidence that these can accurately predict long-term outcomesWHO?HOW?WHEN?WHY?Follow-up:Appropriate length of follow-up to see theeffect on outcome measuresMotivation of patients to complete study!Image in Public domain, courtesy of Wikimedia
84 Statistical Considerations Statisticians are useful for much more than just a sample size calculation!!!Prof. David Spiegelhalter playing with Arco Iris samba band, by Tamara K.This file is licensed under the Creative Commons Attribution-Share Alike 3.0 Unported license (http://creativecommons.org/licenses/by-sa/3.0/deed.en)
85 Defining study objective Defining a study designDefining the primary outcomeSample size calculationStatistical analysisCohort studyCross sectional studyCase-control studyRandomised controlled trialType of outcomeContinuousBinaryClinically relevant and relevant to patientsMeasurableAccurateEffect sizeClinically significantResults in a change in practiceBased on pilot study/previous evidenceCorrect and appropriate for the study design chosenAccount for loss to follow-upWhich patients to include in the analysisE.g. principle of intention to treat in randomised controlled trialsAccount for confoundingMissing dataSubgroup analysisMultiplicitySensitivity analysis
86 Sample size State null hypothesis and choice of statistical test Is the detectable effect size likely to be clinically significant and result in a change in practice?Is the sample size based on previous evidence / pilot results?Are the sample size calculations correct and appropriate for the study design chosen?Has loss to follow-up been accounted for in the sample size?
87 Common statistical errors in applications Statistical analysis does not meet objectives or type of data collectedSection is too ‘general’ ‘continuous data will be analysed using t-tests’Copied from other/ previous grant application without tailoringNo costing for statistical support
88 Outcomes / AnalysisMeasurements accurately reflect the impact of the intervention or are surrogate / proxy measures (and if so, evidence that these can accurately predict long-term outcomes)Appropriate length of follow-up to see the effect on outcome measuresIf there is a long period of follow-up, how will the patients be motivated to complete the study?Protection of outcome measurement from biasSub-analysis clear and documented – rationale and designListing of confounding factors – measured / factors for stratification with rationaleSet out analysis plan logically – descriptive tabulations, approach to analyzing associations among variables, qualitative analytic methods
89 What is Health Economics? The NHS operates in a world of scarce resources (e.g. MONEY, labour, medicines, hospital beds)Health economics informs how scarce resources may be allocated among treatment, prevention & diagnosis of different diseases. At the extreme:Karl Marx: State allocates resourcesAdam Smith (man in £20 note): Free marketImages in Public Domain. Courtesy of Wikimedia.
90 How do we prioritise resources to treat & prevent disease? Need (how do you measure it / competing needs)Discrimination (e.g. by age, smokers …)Personal merit and social esteemLotteryAbility to pay (free market approach)Do you think any of these methods would work in the NHS consistently?
91 In the NHS, new interventions prioritised if: they can be shown to be effective and fit for purposeAND GOOD VALUE FOR MONEY – i.e. the intervention is likely to be cost-effective (mainly through NICE)Image from Office Online
92 Health economics can help us understand the costs and benefits of different interventions: For example:If the health benefit generated by a new intervention is worth the additional costIf an intervention is more effective than its alternative, but its related costs are exorbitant, this intervention will represent poor value for money and not a good use of NHS resourcesIf one intervention provides more health outcomes for the same level of cost this intervention should be adoptedUK research funders increasingly value applications that answer health economic questions
93 Planning a project“Beagle 2 could have crashed into Mars because the atmosphere on the planet was less dense than expected”, explained Professor Pillinger. “The probe may have been going too fast for its parachute and airbags to bring about a soft landing.”But contrary to belief, the mission was not a failure. The technology used for the project could play a major part in managing the tuberculosis pandemic.(Open University WebSite, March 2010)Photograph: ESA
94 The materials, equipment and space you need The time you need The people you needThe materials, equipment and space you needThe time you needHow you organise the aboveWho?What?When?Where?For how long?Think about what will need to be done on a day to day basisThink about maximising benefits to all involved
96 Various roles, for example: CollaboratorsSteering committeeDay to day project manager/ co-ordinatorResearch assistantsAdministrative staffSupport staffConsultantsData Safety Monitoring BoardPatients/ public
97 CollaboratorsBring the right skill mix to the research teamVarying degrees of involvementSteering committeeOverseeing the work, ensuring that the study happens asintendedDay to day project manager/ co-ordinatorManaging the study on a day to day basisResearch assistantsData collectionCollection and processing of samplesAdministrative staffAdministrative support
98 Support staffPeople external to the study, but whose help may be needed.E.g. staff on wards, library, ethics committee, local data protection officerConsultantsFor specialist help and adviceE.g. statisticians, economists, computingData Safety Monitoring BoardIndependent ; periodically check on data being collected for patient safety purposes; broad-ranging monitoring of data quality and integrityPatients/ publicDifferent levels of involvementE.g. Consultation, collaboration, user control
99 Staff management: very important Organising the personnelWho is in charge of what?Must be clearly designated at the startEveryone involved must be committed in terms of interest and timeStaff management: very important
100 Logistical issues in recruitment Photograph: NASA
101 Staff Sites Participants Liaison with personnel Development of job descriptionsAdvertisementsInterviewingGetting proper referencesSitesLiaison with managementLiaison with health professionalsInformation about the study, what it will entail for the study site,the information being collected etc.ParticipantsLook at throughput/ numbers to give you an idea about how long it will takeFirst contact, informed consentGive yourselftime!
102 Recruiting Participants Clear recruitment plan and sampling strategy (note any key groups of patients likely to be excluded)Realistic timing - use local data and apply eligibility criteria, refusals, variation in caseload etcMake effort to ensure inclusion of minority / disadvantaged groups / range of patients across all socioeconomicgroups– describe provision for overcomingpoor literacy and language barriersImage: ddpavumbafreedigitalphotos.netImage from Office Online
103 The importance of communication Photograph: NASA
104 Before the start of the study Who should know about the study?Who should be involved?Formal agreements (e.g. sponsorship, financial arrangements, data ownership and authorship)External communication (e.g. Websites, Press Office)Examples of who needs to know:Trust R&D officesOther senior managersEthics committee(s)Clinicians caring for patients involvedNursing staff if wards involvedComprehensive Local Research NetworksMedical records officerClinic/ward clerksGPs
105 On-going communication Clearly defined roles between researchers – ‘chain of command’, appropriate supportTo ensure data quality, reaching of project milestonesWith ethics committee (any change in protocol, methods, dilemmas etc.)With participants: contact person for everyone, whenever required (e.g. everyday, or ‘on-call’, as necessary)Back-up informationWith study sites and personnel: politics and facilitationClearly identified researchers on wards etc.General information exchangeQuery-answeringTrouble-shootingFeedback and acknowledgement
107 Miscellaneous, e.g. staff training StationeryPostageTestsEquipmentTelephonesTravelPrintingMiscellaneous, e.g. staff trainingPhotos: Wikimedia
108 Logistical considerations in study design Photograph: NASA
109 Assigning responsibility and ensuring time for these Can your protocol be operationalised as a guidance document for staff?Sample selection and monitoring recruitmentIf applicable:RandomisationArranging an interventionAssigning responsibility and ensuring time for these
110 Logistical considerations in data collection & data management Data collection on MarsPhotograph: NASA
111 Types of tools Logistics Measuring outcomes – tools need finding (N.B. permissions) or developing (N.B. validity, reliability)Lay and Non-English speaking issuesCollecting other information, e.g. process, administrative, contextualCoding for the computer, development of databaseOnly collect what you need to answer the questionLogisticsHow will the data be collected: By whom? Where? When?Holidays, bank holidays (postage)Data checking and quality controlConfidentiality/ data protection issuesProcedures for reporting adverse eventsData management and storage (e.g. double blind entry, restricted access, back-up, encryption etc)Computing – coding and data entryData cleaning
113 The importance of piloting Always pilot!It checks that everything works:Management and personnelMethodsFeasibility – technically, politically, logistically, methodologicallyTraining of fieldworkersSampleTiming and content of questionnaires/ interviewsValidity of tools and data sources (feedback from participants)Overall time-scaleData entry, cleaning and analysis
114 Logistical issues in data analysis Cray 2 Supercomputer, NASAPhotograph: NASA
115 Planned before the study has started: Equipment (hardware, software)Statistical methodsStatistics and computing supportOther analysis issues (e.g. qualitative, health economic)TIME
116 (11) Case for Support – Part 3 Dissemination, Outputs & Expertise Pigeon postSource: Wikipedia (Public Domain)
117 Routes for Outputs/ Dissemination Via:Involved patients / lay members & local interest groupsResearch network meetingsTargeted conferences for presentations / postersTargeted peer-reviewed journals - maximum readership within fieldOther media, as appropriateUse of workshops / training packagesGuidelines which could be altered by the resultsTargeted groups of specialist / professional bodiesInclusion of results in existing meta-analyses / Cochrane reviewsMembers of the team involved in teaching
118 Not forgetting… And logistically… And importantly… Participant patients and staffReport for funding bodyAny reports for Ethics CommitteeAnd logistically…TimePersonnelAnd importantly…How will findings be translated into health care to produce patient benefits
119 Expertise: have you got your A-team? 3412Hypatia (Ὑπατία)1 ) Mad Scientist drawn by JJ, Image provided courtesy of Wikipedia. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.22) The School of Athens - fresco by Raffaello Sanzio. Image in the public domain supplied courtesy of Wikipedia.3) Marie Currie ca Image in the public domain supplied courtesy of Wikipedia.4) This image is available from the United States Library of Congress's Prints and Photographs Division under the digital ID cph.3b Image in the public domain supplied courtesy of Wikipedia.
120 Questions a funder will ask: What are the specific roles of members of the research team?Does the team:Have appropriate track records in research?Demonstrate appropriate skills mix - clinical and research experience?Include, where appropriate, a statistician, health economist?Include a PPI representative?Have genuine interest and time to commit to your researchOverall, why is this group of people best placed to do the work?
121 (12) Management and Governance Research timetable: milestones and deliverablesManagement arrangementsPrevious relevant workMeasures of success, risks and how to manage themEthical issues
122 EthicsImage from Office OnlineDoes the research involve specific areas with particular ethical implications?Ethics committee approval required?Has the proposal already been granted a favourable opinion from relevant regulatory authorities?
123 (13) Intellectual Property and Innovation Current IPR held by applicants and relevance to current applicationHas a search for existing IPR been undertaken?Related/ similar IPR which may affect freedom to operate?Will IPR be produced or improved through the research?Management of any IPRCompetitive devices, practices and technologies and advantages over these of any likely to be produced through the research?Benefits of any likely IPRRegulatory considerations and risks to be considered for any new technology to reach the market?
124 (14) NIHR Infrastructure Links to NIHR networks - look atRDS inputInvolvement with other NIHR partnersImage from Office Online
126 Value for moneyJustification for major sources of expenditure, e.g. staffJustification for how research costs have been allocatedJustification for how NHS Support and Treatment costs will be allocated.
127 The people you will need to involve The local NHS R&D Finance Officer of your employerIf you are collaborating with other organisations, the appropriate Finance Officer in each oneYour local Comprehensive Local Research Network (for any NHS Support Costs required)Photo: renjith krishnan,
128 The RfPB Finance FormAlways: Consult your Finance Office at an early stageBe aware:That costs are broken down by project yearOf timings for university/ trust/ CLRN approvalOf which organisation is taking responsibility for the project – their authorised signatory will have to sign off the form.That if the study is a collaboration between organisations, a ‘system’ will not necessarily exist to sort out respective costsThat the participation/ costs of other collaborators will also have to be signed off by themThat, if successful, formal collaboration agreements will need to be in place before funds are distributedPhoto: renjith krishnan,
129 Directly Incurred Costs Q. What is a directly incurred cost?Costs of people/ other things that are being employed/ used exclusively for the projectResearch, technical and other staffTravel and subsistenceEquipmentConsumablesPPIOther, e.g: Conference fees, room hire, publication, trainingPhoto: renjith krishnan,
130 HEI Indirect Costs –Estates Charges What are HEI Indirect Costs – Estates Charges?For universities only, the estate costs for staff, i.e. physical environment required for general staff support.E.g. For Oxford University, for 2010/11, the estate charges, per full time member of staff per year, regardless of grade, are £3,367 (£12,020 for lab. (scientific) departments)Photo: renjith krishnan,
131 HEI Indirect Costs – Other Indirect Costs What are HEI indirect Costs –Other Indirect Costs?For universities only, (i) the overheads associated with employing someone, (ii) the costs of people already employed taking ‘time out’ for the purposes of the study (e.g. supervision), & (iii) university-based technical support (e.g. laboratory costs)E.g. For Oxford University, for 2010/11, the additional costs (or ‘overheads’) associated with employing a full time member of staff, regardless of grade, is £44,124.Photo: renjith krishnan,
132 Commercial/ Other Partner Organisation Indirect Costs Q. What are ‘Commercial/ Other partner Organisation Indirect Costs’?The costs of resources used by the research that are shared by other activities.Seek advice from Finance DepartmentPhoto: renjith krishnan,
133 NHS Overheads E.g. Estates charges Telephone rental etc. Q. What are ‘NHS Overheads’?Costs of resources used by a project, but shared by other activitiesE.g.Estates chargesTelephone rental etc.Photo: renjith krishnan,
134 NHS Support Costs Extra patient tests Extra in-patient days Q. What are ‘NHS support costs’?The additional patient care costs that result from the research being run in an NHS organisation. These costs would end when the project ends.Extra patient testsExtra in-patient daysExtra staff time, e.g. consultants or nurses taking consent, extra attention due to the researchThis funding would usually come via the Comprehensive Local Research Network, and is not included in the total of the grant application figurePhoto: renjith krishnan,
135 NHS Estimated Treatment Costs Q. What are ‘estimated treatment costs’?The treatment costs that relate to the research intervention.You are asked for information on the research treatment (intervention), and the normal standard treatment, so the additional costs for running the research intervention can be calculated.These costs would normally come from the NHS as part of normal patient care commissioningDifferent treatment, or treatment administered in a new way, or a new place.Photo: renjith krishnan,
136 Costing peopleResearchers involved with the study (but already in post)Researchers employed exclusively for the study, e.g. experienced day to day study/ clinical trial manager/ co-ordinator/ primary researcher, research assistantsCollaboratorsSteering committeeAdministrative staffSupport staffConsultantsData Safety Monitoring Board (DSMB)Patients and public involved with research (PPI)Photo: renjith krishnan,
137 Regular staff (internal) Consultancy (external) Salaries:Regular staff (internal)Consultancy (external)Basic salary (check scales)National insuranceSuperannuationIncrements/ pay risesIndirect costs (includingEstate costs)Daily rateRate per sample etcHave you considered dates for each major phase of the study, with staff responsiblePhoto: renjith krishnan,
138 Costing other resources StationeryE.g. How many questionnaires, how long, how many times; invitations; letters, information sheets, how to represent study to public/ patientsPrintingE.g. How many questionnaires, how long, how many times; invitations; letters, information sheetsPostageE.g. How many mailings, follow-ups and remindersTravelE.g. researcher to participant, participant to researcher, meetings to sites to set up and run project, dissemination, PPI, Steering Committee meetingsPhoto: renjith krishnan,
139 E.g. Clinical tests, laboratory tests Equipment E.g. monitoring instruments, computing equipment (including software), desks, chairsTelephonesE.g. Office and mobile; standard rates for institutions?MiscellaneousE.g. Staff training, running costs of equipment, advertisements for staff, incentives (within ethics guidance)Photo: renjith krishnan,
140 Give yourself some leeway Overall budgetHave you considered:How the resources requested will be usedHow realistic and accurate the costings areWhether the ‘best price’ has been found for tests etc?Whether anything appears ‘over-budgeted’?Whether the requested resources are justified?Does the proposal represent the most efficient use of resources?Is the research good value for money?Give yourself some leewayPhoto: renjith krishnan,
141 (16) Monitoring information Information used by DoH and NIHR to monitor and categorise applicationsRDS use and satisfaction with serviceCarbon guidelines(17) Suggested ReviewersSuggest three potential peer reviewers who have relevant expertiseMay or may not be used
142 (18) Supporting Documentation E.g.Flow diagramGantt ChartBibliography - ensure that all papers cited are listed clearly and logicallyLetters of support
143 The inclusion of a Gantt Chart is mandatory 31st FW Balkan Operations Gantt chart. United States Air Force, 2000Image in public domain courtesy of Wikimedia
144 (19) Declarations and Signatures Lead ApplicantHost Organisation (finances)Host Organisation (Head of Department/ Senior Manager)Give yourself TIME to get these!renjith krishnan, freedigitalphotos.net
145 Part 4 Top Tips: Optimising your chances of success (Inspired by RfPB feedback)
146 Make sure: Your study is in scope You submit your proposal on time It is complete!It is a ‘well-crafted’ proposalYour proposal is understandable to a diverse audienceYou have shown the relevance of the study to the NHSYou have had adequate lay involvementYou have the right skill mixYou have made the necessary research partnerships
147 Make sure:You have demonstrated a need for the study, and your knowledge of the areaAny necessary pilot work is undertaken or planned appropriatelyInterventions (e.g. for RCTs) are clearly describedYou have an appropriate study designOther aspects of the methods and procedures are clearYou have consulted appropriate experts and incorporated their suggestionsYou have given yourself a realistic timetableYour study is sensibly fundedDon’t be over-ambitious!
148 Any questions? MANY THANKS! EvaluationPhoto: Simon Howden, FreeDigitalPhotos.netFreeDigitalPhotos.net staffFreeDigitalPhotos.net