1. S CORING I NTERNATIONALITY O F J OURNALS V IA S TATISTICAL E STIMATION Snehanshu Saha CBIMMC & CSE PESIT South Campus snehanshusaha@pes.edu

2. How (NOT) to measure scientific impact? A Modeling approachMTech-CSE PESIT- BSC C ONTENTS  Introduction  Challenges/Misconceptions  Existing system  Proposed system  Analytical/Computational model—”work in progress” 2

3. How (NOT) to measure scientific impact? A Modeling approachMTech-CSE PESIT- BSC I NTRODUCTION  The impact factor (IF) of an academic journals is a measure which reflects the average number of citations to recent articles published in the journal.  It is frequently used as relative importance of a journal within its field, with journals with higher impact factors considered to be more important than those with lower ones.  The impact factor is highly dependent on the academic discipline, possibly on the speed with which papers get cited in a field 3

4. How (NOT) to measure scientific impact? A Modeling approachMTech-CSE PESIT- BSC I NTRODUCTION  The Web of Science (WoS) and Scopus are the two main citation databases that are frequently used to rank journals in a discipline in terms of their productivity as well as the total citations received so as to indicate the journals impact, influence.  The most powerful indices used by these databases to rank journals are the journal impact factor (JIF) used by WoS, & SCImago Journal Rank (SJR) used by Scopus  Both the JIF and SJR are commonly used to rank, evaluate, categorize and compare journals 4

5. How (NOT) to measure scientific impact? A Modeling approachMTech-CSE PESIT- BSC C HALLENGES  The impact factor is not always a reliable instrument, it is used for measuring and comparing the influence of entire journals, but not for the assessment of single papers, researchers or research programmes  Calculation of impact factor based on various factors  To find the significant factors  Assignment of weight to each significant factor 5

6. How (NOT) to measure scientific impact? A Modeling approachMTech-CSE PESIT- BSC E XISTING S YSTEM 6

7. How (NOT) to measure scientific impact? A Modeling approachMTech-CSE PESIT- BSC E XISTING SYSTEM  Where SJRi -Scimago Journal Rank of the Journal i. Cji - Citation from journal j to journal i. Cj - Number of References of journal j. d - Constant, normally 0.85. e - Constant, normally 0.10. N - Number of Journals Artj - Number of Articles of journal j 7

8. How (NOT) to measure scientific impact? A Modeling approachMTech-CSE PESIT- BSC EXISTING SYSTEM-redundancies  More factors are included,so data availability will be relatively difficult, and the evaluation cost will increase  Process is iterative so if initial chosen value is wrong it will take more time to calculate the SJR 8

9. How (NOT) to measure scientific impact? A Modeling approachMTech-CSE PESIT- BSC P REVIOUSLY HOW IT IS DONE ?  The impact factor is based on the arithmetic mean number of citations per paper, yet citation counts follow a power law distribution and therefore the arithmetic mean is a statistically inappropriate measure 9

10. How (NOT) to measure scientific impact? A Modeling approachMTech-CSE PESIT- BSC P ROPOSED SYSTEM  Journals will be classified as National or International based on Internationality Index from Science & Technology DBs.  Calculating various Factors and then finding the correlation between them.  Correlation will help in dimensionality reduction.  Journal name and ISSN will be the input to GUI which will return the internationality score 10

11. How (NOT) to measure scientific impact? A Modeling approachMTech-CSE PESIT- BSC Workflow  Finding the correlation between various factors  Assigning appropriate weight to each significant factor  Designing a GUI  Implementation  Testing 11

12. How (NOT) to measure scientific impact? A Modeling approachMTech-CSE PESIT- BSC I NPUT DATA  Source: http://www.scimagojr.com  Criteria : Below Criteria is applied to download data for analysis  Complete Data Sheet: 12

13. How (NOT) to measure scientific impact? A Modeling approachMTech-CSE PESIT- BSC E XISTING F ACTORS 1. H index:- No. of articles(h) that have received at least h citation over the whole period 2. Total Documents:- Published Document in particular Journal 3. Total Docs (3years):- Published Document in the 3 previous years in particular Journal 4. Total References:- Bibliographical Reference in Journal 5. Total Cites (3years):- Citation received in a year for journal documents published in previous 3 years 6. Self Cites (3years):- Journal's self citation in year to its own documents published in previous 3 years 7. Citable Docs. (3years):- Citable documents published in previous 3 years 8. Cites / Doc. (4years):- Average Citation per document in a 4 year period 9. Cites / Doc. (3years):- Average Citation per document in a 3 year period 10. Cites / Doc. (2years):- Average Citation per document in a 2 year period 11. References / Doc.:- Average references per document 12. Cited Docs.:- Number of Document published in previous 3 years that have been cited at least once 13. Uncited Docs.:- Number of Document published in previous 3 years that have never been cited 14. % International Collaboration:- Publication Ratio whose affiliation include more than one country 13

14. How (NOT) to measure scientific impact? A Modeling approachMTech-CSE PESIT- BSC O PTIMIZED F ACTORS FOR STATISTICAL ANALYSIS 1. H index:- No. of articles(h) that have received at least h citation over the whole period 2. Total Documents:- Published Document in particular Journal 3. Total Docs (3years):- Published Document in the 3 previous years in particular Journal 4. Total References:- Bibliographical Reference in Journal 5. Total Cites (3years):- Citation received in a year for journal documents published in previous 3 years 6. Citable Docs. (3years):- Citable documents published in previous 3 years 7. References / Doc.:- Average references per document 8. Cites / Doc. (2years):- Average Citation per document in a 2 year period 14

15. How (NOT) to measure scientific impact? A Modeling approachMTech-CSE PESIT- BSC R EGRESSION ANALYSIS – P HASE 1 1. Tool Used:- Excel Data Analysis Module 2. Results:- 15

16. How (NOT) to measure scientific impact? A Modeling approachMTech-CSE PESIT- BSC R EGRESSION ANALYSIS – P HASE 1 Inference:- 16 FactorP-ValueCorrelation Coefficient Optimization Decision H index 0.0050.69 Total Docs. (2012) 0.0120.16 Total Docs. (3years) 0.3340.37 Total Refs. 0.0130.40 Total Cites (3years) 0.0010.55 Citable Docs. (3years) 0.5160.37 Cites / Doc. (2years) 0.0000.85 Ref. / Doc. 0.7510.17

17. How (NOT) to measure scientific impact? A Modeling approachMTech-CSE PESIT- BSC R EGRESSION ANALYSIS – P HASE 2 1. Regression and Residual Analysis Performed for Remaining Factors 2. Regression Results:- 17

18. How (NOT) to measure scientific impact? A Modeling approachMTech-CSE PESIT- BSC R EGRESSION ANALYSIS – P HASE 2 Residual Results:- 1. Data points with extreme +ve Residuals 18 2. Data points with extreme -ve Residuals  These data points are 10% of overall Data and removed for further statistical and regression analysis

19. How (NOT) to measure scientific impact? A Modeling approachMTech-CSE PESIT- BSC R EGRESSION ANALYSIS – P HASE 3 1. Tool Used:- Excel Data Analysis Module 2. Results:- 19 Total Docs(3 Years) has P value >0.05 and a negative coefficient which is not logical. Hence, removed from further analysis

20. How (NOT) to measure scientific impact? A Modeling approachMTech-CSE PESIT- BSC R EGRESSION ANALYSIS – F INAL P HASE 1. Tool Used:- Excel Data Analysis Module 2. Results:- 20 R Square is >0.75 F value is <0.05 All P-values are <0.05

21. How (NOT) to measure scientific impact? A Modeling approachMTech-CSE PESIT- BSC R EGRESSION E QUATION Journal Influence Factor = 0.09237 + (0.00585 * H index ) + (0.00042 * Total Documents For Current Year ) - (0.00001 * Total References ) + (0.00012 * Total Cites in previous 3 years ) - (0.00040 * Citable Documents in previous 3 years ) + (0.33174 * Cites / Doc. in previous 2 years ) 21

22. How (NOT) to measure scientific impact? A Modeling approachMTech-CSE PESIT- BSC V ALIDATION 1. Comparison has been done between Journals ranking from 1) Existing system and 2) Newly derived regression equation to validate optimization of factors and accuracy of regression equation. 2. Validation is performed on 2012 journals data of Computer Science Application 3. Results: Average Difference in Journal’s Ranking : 0.12 Median of difference in Journal’s Ranking : 0.08 Difference Group Analysis 22 Difference Group in Journal’s Ranking% of Data points +/- 0.157% +/- 0.278% +/- 0.390%

23. How (NOT) to measure scientific impact? A Modeling approachMTech-CSE PESIT- BSC V ALIDATION WITH A NOTHER A PPLICATION A REA 1. Comparison has been done between Journals ranking from 1) Existing system and 2) Newly derived regression equation to validate optimization of factors and accuracy of regression equation. 2. Validation is performed on 2012 journals data of Computer Networks and Communications 3. Results: Average Difference in Journal’s Ranking : 0.34 Median of difference in Journal’s Ranking : 0.13 Difference Group Analysis 23 Difference Group in Journal’s Ranking% of Data points +/- 0.144% +/- 0.265% +/- 0.376%

24. How (NOT) to measure scientific impact? A Modeling approachMTech-CSE PESIT- BSC T OOL D EVELOPMENT A web based tool has been designed using HTML and Java Scripting to enable calculation of Journal Influence Factor 24

25. How (NOT) to measure scientific impact? A Modeling approachMTech-CSE PESIT- BSC N EXT S TEP  Lets MOVE TO BOARD, :P 25

26. How (NOT) to measure scientific impact? A Modeling approachMTech-CSE PESIT- BSC Acknowledgement Scientometric Modeling Group: Neelam Jangid( former member) Anand Narasimhamurthy Chitra Balasubramaniam Harsha R.S Nandita Dwivedi Avantika Dwivedi