1. Natural Research Assessment: ability and new requirements for CRIS Sergey Parinov, CEMI RAS, Russia, Moscow

2. Natural Research Assessment is … –an inherent part of science communication, since it is a basic consequence of existed labor division and knowledge exchange between scientists –a practical “testing” of research outputs by scientists when they are trying to use it for producing new scientific knowledge –a statistical portrait of total trials-and-errors for some researchers, their results, etc. Can we practically use it? If yes, how?

3. Natural Assessment in Social Systems All social systems, where independent actors are working with labor division and products exchange between them, naturally produce trial-and-error usage statistics If in the system the practical testing of is doing well and trial-and-error data is available for actors, so the system provides for actors useful natural assessment mechanism and statistics By such assessments actors know how their activity are currently valuable for the system and what/whose activities are the most useful now Than better such signaling mechanism works, than the system is more efficient

4. Scientific Social System Features Science is a social system with labor division and knowledge exchange between scientists in a trial- and-error form Trial-and-error usage statistics are mostly latent since researchers are testing “units of thought” hidden inside publications which are currently visible form of scientific exchange Scientists can get trial-and-error data mostly in a form of lists of publications selected by publishers and citations, what is really not enough Current scientific practical testing (trial-and-error) process is doing not very well and compare with many other social systems the Science has weaker signaling system and its efficiency can be improved

5. Current Role of CRIS (e.g. socionet.ru CRIS) CRIS is an interface for scientists to give their research outputs for common usage and to take “products” from other scientists for their personal usage CRIS is working as a scientific circulation mechanism, since it delivers “products” from “producers” to “consumers” CRIS is a tool of practical testing of research outputs and virtual environment for scientific trial-and-error acts

6. Specification of Tasks Strong natural assessment and professional signaling system can be designed by: –discovering of what is a matter (object) of scientific trial-and-error process in producing a new knowledge –efficient scientific circulation mechanism, which involves all objects into representative trial-and-error process –registering and collecting all existed trial-and-error data with presenting them as publicly available/visible statistics –the statistics should show both side of scientists’ activity within a network of professional relations: as a “producer” and as a “consumer” of research outputs We should exploit typical motivations and scientific behavior models: a wish to get professional recognition and existed norms to provide professional reviews

7. Trial-and-error process within CRIS An author registers research output as a ready for testing scientific object-for-reuse (OfR) by –specifying which research materials were used as roots/basements for his/her output (citation links) –specifying materials/scientist where/by whom the output could be used/reviewed (links to possible users) Authors of linked materials receive a notification about created links, and –they can protest on how the materials were used –they can use suggested OfR and link it with their materials, or can review it, or can ignore it

8. Scientific Communications within CRIS On a producer side –specify used OfR with quality characteristics –request on using- reviewing own OfR by linking it with other OfR or scientists On a consumer side –protest against usage characteristics and/or provide comments on it, or do nothing –ban requests from some authors, or specify personal reviewing rate, or rewrite own OfR by using/citing suggested OfR

9. Trial-and-error statistics within CRIS Accumulated data about scientist-producer activity –number of produced OfR for certain period of time –number of OfR usage for certain period of time, including quality characteristics distribution –number of OfR reviews for certain period of time, including quality characteristics distribution Accumulated data about scientist-consumer activity –number of requests from scientists on using/reviewing their OfR –number of made citations (utilized OfR), including quality characteristics distribution –number of made reviews, including quality characteristics distribution –number of banned authors, queue length of requests compare with personal rate, number of rejections

10. Who will benefit? A scientist – –by data on how people used his/her outputs and by usage request as some can be valuable A research institution – –by data on staff’s produced/used outputs, etc. A research funder – –by data on produced/used outputs, performance statistics about researchers and organizations A research community – –by better scientific circulations, usage and professional signaling system

11. Conclusion We are implementing natural research assessment scheme within Socionet CRIS (socionet.ru) as a pilot to allow a playing with it We suggest for euroCRIS to include a support of natural research assessment into CRIS-CERIF model Thanks!

12. Necessary Role of CRIS Electronic registration research outputs/results focused on its usage Tools to request usage/review/evaluation from a community and/or its members A form to make usage/review/evaluation in easy way Taking statistics on research performance of scientist in a role of a producer and a consumer research outputs/results

13. Requirements for new CRIS features To operate with OfR as citation and artifact information objects To make semantic links (by usage/impact metrics) with CRIS objects (material, person, etc.) To run monitoring of linkages and notification services To give scientist an ability to specify a personal norm of reviewing efforts To gather data, collect and process trial-and- error statistics

14. Background Open Science as a conceptual platform –1 open access to research –2 open access to usage and impact data –3 open access to basic assessment data CRIS Socionet as technological platform –open archives and IR –scientific circulation mechanism –monitoring and statistics services