Presentation on theme: "ASSESSING THE NATIONAL ENTRIES Brian Walker"— Presentation transcript:
ASSESSING THE NATIONAL ENTRIES Brian Walker email@example.com
Objective of the assessment The Assessment Process The procedure to be adopted Timescales Accessing the entries Examples and discussion Outline
To review the entries and “validate” the self assessments to enable a short list to be selected. Objective To provide an independent view of the entrant’s maturity level to assist them in improving their operation Primary Objective Secondary Objective
Process The companies Self-Assess their maturity level against a series of statements and provide evidence to support this level Assessors check the companies assessment against the evidence they provide Assessors’ role is to check that the maturity level selected looks reasonable, i.e. not obviously wrong The Assessors do not rescore the entry from scratch based on the evidence provided
Assessment Aims The aim is to select a short list of companies for each category We need confidence that the best entry in that category will be among the companies short-listed Companies differ and the score alone, while an indicator, may not provide the complete picture
Comments Comments, both on individual questions and overall will be very helpful To explain any scoring changes To assist with shortlisting To focus on appropriate issues during the review visit To provide feedback to the company
What information will Assessors have? Companies will indicate their assessment by selecting one of 5 statements showing a progressively higher maturity level They will also provide evidence to support this selection by ticking relevant statement(s) in relation to the core questions and by providing free text evidence The amount of evidence that companies can provide is limited, hence full proof of their own assessment is not possible or expected
Assessment Procedure 1 Read the general information first to get a feel for the company and its operations Check each question, noting the maturity level selected Check whether the evidence is reasonably supportive of the maturity level selected
Assessment Procedure 2 The lowest two maturity levels require no written evidence to support them Higher maturity levels should be supported by evidence that is relevant and indicates that the maturity level has been achieved The highest level needs more specific justification rather than general claims Note the maturity level statements are generic and may not match every company. Sensible parallels need to be drawn in some cases.
Assessment Procedure 3 Where the evidence generally supports the maturity level selected – no action required For higher maturity levels some evidence is needed but don’t look for complete confidence Reduce score by one level where evidence is clearly inadequate – add comment to explain In exceptional cases reduce further but only where evidence clearly indicates a lower level – again comments should explain reasons Companies do score themselves too low so be prepared to raise scores when appropriate – comment should explain reason
Assessment Procedure 4 “Not applicable” (N/A) is an option for all questions Entrants need to explain why the statement is “Not Applicable” if selecting this Not Applicable means that it is not relevant to the type of business and its operations It is not a valid response where the company has no system when a similar “best practice” company would have As previously in MX if N/A is accepted the question is ignored in calculating percentages Assessors need to check that the reason given is valid and N/A is a reasonable response Not Applicable responses
Assessment Procedure 5 Two or three Assessors will be asked to assess each entry independently A Lead Assessor will compare the assessments and adjudicate where necessary to produce an assessed report and score The MX Board will review the top scoring entries, including any comments from Assessors to determine a short-list of companies to be visited.
Timescales The Awards have now been launched The final date for entries is 23 May 2014 Entries will sent for assessment as received Few expected much before closing date Assessors need to complete assessment by 13 June 2014, earlier if possible The Short-list will be announced by 20 June 2014 Vital that if Assessor is unable to undertake an allocated Assessment they notify staff immediately
The companies you are to assess listed here Assessor 1 & 2 here when assigned. Click to select company Assessor’s Entry Sheet
Click to access statements Click to see summary report
Initial Screen for Company Takes you to the questions Details about company
Initial Screen for Company Do not click back from this page or questions
Initial Screen for Company Click Return to Administration to go back
Example showing Maturity Level, Pre-set Evidence and Free text Evidence
Entry Options As with previous MX companies can enter for overall or individual categories Category entry consists of some Core questions plus some additional questions To be eligible for overall winner, company must complete all Core questions plus some categories If entering only Categories then only the relevant questions need to be answered
Core questions attempted Category attempted Drop down to select What have companies entered?
Entry Screen for extra question for MX Category Suggestions for evidence – no pre-sets 2000 characters for evidence instead of 1000 Tabs go to contributing questions from Core set
Core questions retain pre-set evidence Also allow 2000 character evidence
Assessor’s view of Statement Page Category notes apply to whole section so comment must indicate statement it applies to: “e.g. SM 1 No financial benefits evidenced, 5 reduced to 4” Assessor’s comments go here Assessor changes maturity level if reqd.
Key Points Change higher maturity levels only if clearly not supported by the evidence Be prepared to increase score where justified But don’t rescore entry just on the basis of the evidence Comments are invaluable whether maturity level changed or not Make comments brief, specific and constructive