Presentation is loading. Please wait.

Presentation is loading. Please wait.

The role of the Peer Reviewer for the New Zealand National Standards Scheme Nga Kaupapa Whaimana o Nga Whare Taonga o Aotearoa.

Similar presentations


Presentation on theme: "The role of the Peer Reviewer for the New Zealand National Standards Scheme Nga Kaupapa Whaimana o Nga Whare Taonga o Aotearoa."— Presentation transcript:

1 The role of the Peer Reviewer for the New Zealand National Standards Scheme Nga Kaupapa Whaimana o Nga Whare Taonga o Aotearoa

2 Focus on the process of the Peer Review activity My involvement stems from a request from NS Te Paerangi for expressions of interest from the sector to become a peer reviewer of the New Zealand Standards Scheme and I have beenMy involvement stems from a request from NS Te Paerangi for expressions of interest from the sector to become a peer reviewer of the New Zealand Standards Scheme and I have been Briefing session and training opportunityBriefing session and training opportunity Pairing up and selection of organisations to be peer reviewedPairing up and selection of organisations to be peer reviewed Desk review of provided documentation on organisations undertaking the Standards SchemeDesk review of provided documentation on organisations undertaking the Standards Scheme Site visit by peer reviewersSite visit by peer reviewers Report and summary written and provided to Te Papa National Services Te Paerangi and forwarded to participating organisationReport and summary written and provided to Te Papa National Services Te Paerangi and forwarded to participating organisation

3 Structure of the briefing meeting History of the regionally funded and supported program of the Standards Scheme, giving a profile of the organisations involved, their location, governance/management structures and collection focusHistory of the regionally funded and supported program of the Standards Scheme, giving a profile of the organisations involved, their location, governance/management structures and collection focus In a regional partnership proposal with Te Papa National Services all standards projects must be complete within the financial year allocatedIn a regional partnership proposal with Te Papa National Services all standards projects must be complete within the financial year allocated Pairing up of suitable peer reviewers with some understanding of the organisational structure/focusPairing up of suitable peer reviewers with some understanding of the organisational structure/focus Discussions regarding the process of coordinating the desk and site review including potential timing and travel arrangementsDiscussions regarding the process of coordinating the desk and site review including potential timing and travel arrangements Examples of how to best use time on-siteExamples of how to best use time on-site Examples of scenarios that could occur (experiences brought to the activity through previous peer review encounters)Examples of scenarios that could occur (experiences brought to the activity through previous peer review encounters) Significant time spent with peer reviewer partner on planning for the review of the organisations selectedSignificant time spent with peer reviewer partner on planning for the review of the organisations selected

4 Written documentation Compulsory Module One Governance, Management and Planning, and any other modules from 2 – 5 reviewed. Minimum of 2, maximum 5 modules. The process is made more difficult the more modules are undertaken and lengthier the larger the organisationCompulsory Module One Governance, Management and Planning, and any other modules from 2 – 5 reviewed. Minimum of 2, maximum 5 modules. The process is made more difficult the more modules are undertaken and lengthier the larger the organisation Tests the evidence of museums progress towards achieving/attaining standardsTests the evidence of museums progress towards achieving/attaining standards Tick box with documentation to be viewed on-site (rather than sending all documentation with the submission)Tick box with documentation to be viewed on-site (rather than sending all documentation with the submission) There is an ease of use for the process as all modules available on disc/attachments to so that all annotations by museums and peer reviewers can be done electronicallyThere is an ease of use for the process as all modules available on disc/attachments to so that all annotations by museums and peer reviewers can be done electronically All paperwork (including peer reviewer copies with annotations) returned to reviewed organisationAll paperwork (including peer reviewer copies with annotations) returned to reviewed organisation

5 Site visit Not possible to conclude a Standards assessment purely with a desk review of provided documentationNot possible to conclude a Standards assessment purely with a desk review of provided documentation Desk review raises questions that a site visit is able to answerDesk review raises questions that a site visit is able to answer Discrepancies between written and oral submissions can be clarified and an experience of the way in which the organisation is operating is provided through the visitDiscrepancies between written and oral submissions can be clarified and an experience of the way in which the organisation is operating is provided through the visit Two reviewers gives a variety of perspective and minimise the risk of biasTwo reviewers gives a variety of perspective and minimise the risk of bias The per reviewer partnership is a collegial relationship within a bicultural frameworkThe per reviewer partnership is a collegial relationship within a bicultural framework Often some gap between written material and site visit reality (although this can be simply a matter of interpretation)Often some gap between written material and site visit reality (although this can be simply a matter of interpretation) Breaks isolation and helps to build networks for both partiesBreaks isolation and helps to build networks for both parties

6 Site visit Organised to fit a framework of meeting people, touring the facility and discussing the information provided under the Standards SchemeOrganised to fit a framework of meeting people, touring the facility and discussing the information provided under the Standards Scheme There is a formal approach to the organisation to agree on (firstly the peer reviewers named to undertake the peer review) timing of the visit and structure of the visit, who the peer reviewers would like to have contact with (essential that this is the person who has filled out the documentation), and a time that the peer reviewers need to be away from the organisationThere is a formal approach to the organisation to agree on (firstly the peer reviewers named to undertake the peer review) timing of the visit and structure of the visit, who the peer reviewers would like to have contact with (essential that this is the person who has filled out the documentation), and a time that the peer reviewers need to be away from the organisation Timing does often require organisation at museum end (volunteer organisations particularly that might not open every day)Timing does often require organisation at museum end (volunteer organisations particularly that might not open every day) As peer review teams usually undertake 2 reviews the time to leave is often crafted around the need to be somewhere else, when in effect this is to dissuade the potential for the peer review to become a self-help session or therapy session for the museum staff or operatorsAs peer review teams usually undertake 2 reviews the time to leave is often crafted around the need to be somewhere else, when in effect this is to dissuade the potential for the peer review to become a self-help session or therapy session for the museum staff or operators

7 Report writing Feedback documentation presented by that organisation indicates where they believe their current standard liesFeedback documentation presented by that organisation indicates where they believe their current standard lies Above standardAbove standard Standard metStandard met Standard almost metStandard almost met Standard not metStandard not met Peer reviewing uses the provided documentation and site survey to assess whether the organisations assessment of their standards is the same as the peer reviewers, and provides, through the summary documentation, areas in which the peer reviewers believe the organisation can improve its performancePeer reviewing uses the provided documentation and site survey to assess whether the organisations assessment of their standards is the same as the peer reviewers, and provides, through the summary documentation, areas in which the peer reviewers believe the organisation can improve its performance

8 Report writing The summary documentation is returned to the organisation (not the writer of the documentation in every case – but in smaller volunteer organisations there is an inevitability that the person who filled out the documentation is the one peer reviewers will meet on site, and the one who will be responsible for making any changes to the organisation identified in the summary documentsThe summary documentation is returned to the organisation (not the writer of the documentation in every case – but in smaller volunteer organisations there is an inevitability that the person who filled out the documentation is the one peer reviewers will meet on site, and the one who will be responsible for making any changes to the organisation identified in the summary documents Any changes (or not) to the improvement of standards becomes the responsibility of the organisation undertaking the standards scheme (there is at this stage no follow up evaluation process to determine whether an organisation has followed up on recommendations in summary documents by the peer reviewers)Any changes (or not) to the improvement of standards becomes the responsibility of the organisation undertaking the standards scheme (there is at this stage no follow up evaluation process to determine whether an organisation has followed up on recommendations in summary documents by the peer reviewers)

9 Report writing Difficulties of report writing include;Difficulties of report writing include; that site visits and travel often mean that peer reviewers do not have the energy to complete the summary documentation on the day of the peer reviewthat site visits and travel often mean that peer reviewers do not have the energy to complete the summary documentation on the day of the peer review that joint report writing at distance once peer reviewers have returned home is a challengethat joint report writing at distance once peer reviewers have returned home is a challenge consistency of evaluation and summary recommendations across the process because of the variety of skills and experience of peer reviewersconsistency of evaluation and summary recommendations across the process because of the variety of skills and experience of peer reviewers being positive and encouraging in the face of adversity, yet being direct about issues that need to be addressed [constructive specific feedback]being positive and encouraging in the face of adversity, yet being direct about issues that need to be addressed [constructive specific feedback]

10 Benefits to the Peer Reviewer Many peer reviewers made positive comments about how worthwhile the visits were at the debriefing workshops and noted their sincere admiration of the commitment and enthusiasm, especially of the small museums, to improve, or in fact become involved in the process...humbled, stunned by what museums achieved with the resources available and positive community support, strangely bizarre to do what they do for no [financial] rewards and they keep doing and doing it – I respect and value their contribution. [The trial of the New Zealand Museums Standards Scheme 1999/2000, Rivers ORegan Lynch]Many peer reviewers made positive comments about how worthwhile the visits were at the debriefing workshops and noted their sincere admiration of the commitment and enthusiasm, especially of the small museums, to improve, or in fact become involved in the process...humbled, stunned by what museums achieved with the resources available and positive community support, strangely bizarre to do what they do for no [financial] rewards and they keep doing and doing it – I respect and value their contribution. [The trial of the New Zealand Museums Standards Scheme 1999/2000, Rivers ORegan Lynch]

11 Benefits to the Peer Reviewer In the [The trial of the New Zealand Museums Standards Scheme 1999/2000, Rivers ORegan Lynch] report All the peer reviewers said that they gained personally from being part of the standards trial. Specific examples included:All the peer reviewers said that they gained personally from being part of the standards trial. Specific examples included: Bicultural aspects – working closely with Maori perspectives and Maori professionalsBicultural aspects – working closely with Maori perspectives and Maori professionals Better understanding of museum issues, particularly governanceBetter understanding of museum issues, particularly governance An opportunity to expand museum networks and professional contactsAn opportunity to expand museum networks and professional contacts How to improve their own institutionHow to improve their own institution

12 Sector representation National Services Te Paerangi asks for expressions of interest from the sector for people to become peer reviewers and provides a training framework enabling more experienced peer reviewers to work with newer recruits to the process of Peer Review under the National Standards Scheme.National Services Te Paerangi asks for expressions of interest from the sector for people to become peer reviewers and provides a training framework enabling more experienced peer reviewers to work with newer recruits to the process of Peer Review under the National Standards Scheme. Peer reviewers from outside Te Papa Tongarewa give legitimacy to the independence of the activity and also assist in establishing ownership of the scheme by the sector as a whole.Peer reviewers from outside Te Papa Tongarewa give legitimacy to the independence of the activity and also assist in establishing ownership of the scheme by the sector as a whole. With the launching of the scheme in November 2002 the sector can undertake modules of the Standards Scheme independently of NS Te Papa and seek an independent or selected peer review process of their resultsWith the launching of the scheme in November 2002 the sector can undertake modules of the Standards Scheme independently of NS Te Papa and seek an independent or selected peer review process of their results

13 Challenges/Future Finding and retaining an adequate pool of skilled peer reviewers, both Maori and non-Maori acceptable to the sectorFinding and retaining an adequate pool of skilled peer reviewers, both Maori and non-Maori acceptable to the sector Peer reviewers commit time away from their organisations to the present scheme and add the activity to a growing pool of individual responsibilities. This can lead to time delays in finalising summary documentation or to achieving outcomes in work within peer reviewers own institution.Peer reviewers commit time away from their organisations to the present scheme and add the activity to a growing pool of individual responsibilities. This can lead to time delays in finalising summary documentation or to achieving outcomes in work within peer reviewers own institution. The potential pool of organisations that want to be involved in the process and the limited financial resources presently available to commit to a national programmeThe potential pool of organisations that want to be involved in the process and the limited financial resources presently available to commit to a national programme The future use of the summary documents as leverage of funding from the public pool (which isnt very deep) and the long-term sustainability of many of the smaller organisations reviewedThe future use of the summary documents as leverage of funding from the public pool (which isnt very deep) and the long-term sustainability of many of the smaller organisations reviewed


Download ppt "The role of the Peer Reviewer for the New Zealand National Standards Scheme Nga Kaupapa Whaimana o Nga Whare Taonga o Aotearoa."

Similar presentations


Ads by Google