Vol. 15 No. 1 (2024)
Articles

Citation metrics and strategic mutations of scientific research: narratives and evidence

Luca Guerra
University of Brescia
Categories

Published 2024-01-15

Keywords

  • Scientific misconducts,
  • Gaming metrics,
  • Scholarly communication,
  • Research assesment,
  • Bibliometrics.

How to Cite

Guerra, Luca. 2024. “Citation Metrics and Strategic Mutations of Scientific Research: Narratives and Evidence”. JLIS.It 15 (1):144-58. https://doi.org/10.36253/jlis.it-538.

Abstract

Following the diffusion of the management model promoted by the New Public Management also within universities, sectors until then endowed with particular protection, such as the academic one, were progressively subjected to new controls and constraints, which gradually took the form of quantitative surveys, with a growing role of citation metrics. These evaluation processes have given rise to various important critical positions at an international level.

If it is true that, according to Goodhart's law, "when a measure becomes an objective it ceases to be a good measure”, it is even more significant to note that when a measure becomes an objective, what is measured ceases to be what it was before. The quantitative measurement of academic performance has in fact triggered forms of gaming such as to alter the very game of scientific research, its purposes and the forms of its sharing. In the article we will carry out a comparison between the forms of mutation and the narratives that accompany them to see to what extent we are legitimate today to talk about the transformation of scientific research into strategic scientific research.

Metrics

Metrics Loading ...

References

  1. Biagioli, Mario. 2016. “Watch out for cheats in citation game”. Nature 535: 201. https://doi.org/10.1038/535201a. DOI: https://doi.org/10.1038/535201a
  2. Biagioli, Mario. 2020. “Introduction”. In Gaming the metrics: misconduct and manipulation in academic research, edited by Mario Biagioli and Alexandra Lippman, 1-24. Cambridge, The Mit press. https://doi.org/10.7551/mitpress/11087.003.0006. DOI: https://doi.org/10.7551/mitpress/11087.003.0001
  3. Biagioli, Mario and Alexandra Lippman, ed. 2020. Gaming the metrics: misconduct and manipulation in academic research. Cambridge: The Mit press. https://mitpress.mit.edu/9780262537933/gaming-the-metrics. DOI: https://doi.org/10.7551/mitpress/11087.001.0001
  4. Brezis, Elise S., and Aliaksandr Birukou. 2020. “Arbitrariness in the Peer Review Process”. Scientometrics, 2123: 393–411. https://doi.org/10.1007/s11192-020-03348-1. DOI: https://doi.org/10.1007/s11192-020-03348-1
  5. Cari-Ann, Hayer, Mark A. Kaemingk, Jason J. Breeggemann, Daniel Dembkowski, and David Deslauriers. 2013. “Pressures to publish: catalysts for the loss of scientific writing integrity?”. Papers in Natural Resources 689. https://doi.org/10.1080/03632415.2013.813845. https://digitalcommons.unl.edu/natrespapers/689. DOI: https://doi.org/10.1080/03632415.2013.813845
  6. Council of the European Union. 2022. “Council conclusions on research assessment and implementations of Open science. Annex 10126/22.” https://data.consilium.europa.eu/doc/document/ST-9515-2022-INIT/en/pdf.
  7. De Bellis, Nicola. 2014. Introduzione alla bibliometria. Roma: AIB.
  8. Dondio, Pierpaolo, Niccolò Casnici, Francisco Grimaldo, Nigel Gilbert, and Flaminio Squazzoni. 2019. “The ‘Invisible hand’ of peer review: the implications of author-referee networks on peer review in a scholarly journal”. Journal of Informetrics 13: 708–716. https://doi.org/10.1016/J..JOI.2019.03.018. DOI: https://doi.org/10.1016/j.joi.2019.03.018
  9. Edwards, A. Mark, and Roy Siddhartha. 2017. “Science is broken. Perverse incentives and the misuse of quantitative metrics have undermined the integrity of scientific research“. Aeon. https://aeon.co/essays/science-is-a-public-good-in-peril-heres-how-to-fix-it.
  10. Fanelli, Daniele. 2020. “Pressures to publish: what effects do we see? In Gaming the metrics: misconduct and manipulation in academic research, edited by Mario Biagioli and Alexandra Lippman, 111-122. Cambridge: The Mit press. https://doi.org/10.7551/mitpress/11087.003.0011. DOI: https://doi.org/10.7551/mitpress/11087.003.0011
  11. Fanelli, Daniele, Matteo Schleicher, Ferric C. Fang, Arturo Casadevall, and Elisabeth M. Bik. 2022. “Do individual and institutional predictors of misconduct vary by country? Results of a matched-control analysis of problematic image duplications”. PLoS One 17 (3) e0255334. https://doi.org/10.1371/journal.pone.0255334. DOI: https://doi.org/10.1371/journal.pone.0255334
  12. Fang C., Ferric, and Arturo Casadevall. 2011. “Retracted science and the retraction index”. Infective Immunology 79 (10): 3855-9. https://doi.org/10.1128/IAI.05661-11. DOI: https://doi.org/10.1128/IAI.05661-11
  13. Figà-Talamanca, Alessandro. 2000. “L’impact factor nella valutazione della ricerca” [Bozza dell’intervento, Bologna 27 giugno 2000]. http://siba.unipv.it/biblioteche/info/SISSCOWEB_A_Fig_Talamanca_L%27impact_factor_nella_valutazione_della_ricerca_SISSCOWEB.htm.
  14. Garfield, Eugene. 1998. “The impact factor and using it correctly”. De Unfallchirurg, 48 (2): 424. http://garfield.library.upenn.edu/papers/derunfallchirurg_v101%286%29p413y1998english.html.
  15. Gaudino Mario, Brice N. Robinson N., Katia Audisio, Mohamed Rahouma, Umberto Benedetto, Paul Kurlansky, and Stephen E. Fremes. 2021. “Trends and characteristics of retracted articles in the biomedical literature, 1971 to 2020”. JAMA international medicine 181 (8): 1118-1121. https://doi.org/10.1001/jamainternmed.2021.1807. DOI: https://doi.org/10.1001/jamainternmed.2021.1807
  16. Hicks Diana, Wouters Paul, Waltman Ludo, De Rijcke Sarah, Rafols Ismael. 2015. “Bibliometrics: The Leiden Manifesto for research metrics”. Nature. 520: 429-431. DOI: https://doi.org/10.1038/520429a
  17. Lacetera, Nicola, and Lorenzo Zirulia. 2011. “The economics of scientific misconduct”. The Journal of Law, Economics, and Organization. 27: 568–603. https://doi.org/10.1093/jleo/ewp031. DOI: https://doi.org/10.1093/jleo/ewp031
  18. Martinson, C. Brian, Melissa S. Anderson, and Raymond de Vries. 2005. “Scientists behaving badly”. Nature. 435: 737-738. DOI: https://doi.org/10.1038/435737a
  19. Merton, K. Robert. 1998. “The Matthew effect in science, II: cumulative advantage and the symbolism of intellectual property”. Isis. 79, 4: 606-623. DOI: https://doi.org/10.1086/354848
  20. Moher, David, Lex Bouter, Sabine Kleinert, Paul Glasziou, Sham Mai Har, Virginia Barbour, Anne-Marie Coriat, Nicole Foeger, and Ulrich Dirnagl. 2020. “The Hong Kong Principles for assessing researchers: fostering research integrity”. PLoS Biology 18 (7). https://doi.org/10.1371/ DOI: https://doi.org/10.1371/journal.pbio.3000737
  21. journal.pbio.3000737.
  22. Muller, Z. Jerry. 218. The tyranny of metrics. Princeton: Princeton university press.
  23. Naujokaitytė, Goda. 2021. “Number of scientists worldwide reaches 8.8M, as global research spending grows faster than the economy” in Science business. https://sciencebusiness.net/news/number-scientists-worldwide-reaches-88m-global-research-spending-grows-faster-economy.
  24. Oransky, Ivan. 2022. “Retractions are increasing, but not enough”. Nature 608: 9. https://doi.org/10.1038/d41586-022-02071-6. DOI: https://doi.org/10.1038/d41586-022-02071-6
  25. Pantziarka, Pan, and Lydie Meheus. 2019. “Journal retractions in oncology: a bibliometric study”. Future oncology 15 (31): 3597-3608. https://doi.org/10.2217/fon-2019-0233. DOI: https://doi.org/10.2217/fon-2019-0233
  26. Power, Michael. 1997. The Audit Society. Oxford: Oxford University Press.
  27. Smith, Richard. 2006. “Peer review: a flawed process at the heart of science and Journals”. Journal of the Royal society of medicine 99 (4): 178-182. DOI: https://doi.org/10.1258/jrsm.99.4.178
  28. Tijdink, K. Joeri, Anton C.M. Vergouwen, and Yvo M. Smulders. 2013. “Publication pressure and burn out among Dutch medical professors: a nationwide survey”. PLoS One 8 (9): e7338. https://doi.org/10.1371/journal.pone.0073381. DOI: https://doi.org/10.1371/journal.pone.0073381
  29. Turbanti, Simona. 2018. Strumenti di misurazione della ricerca. Milano: Editrice Bibliografica.
  30. Wilsdon, James, ed. 2015. The metric tide: report of the independent review of the role of metrics in research assessment and management. Thousand Oaks: SAGE publications. https://doi.org/10.13140/RG.2.1.4929.1363. https://sk.sagepub.com/books/the-metric-tide. DOI: https://doi.org/10.4135/9781473978782
  31. Wouters, Paul. 2020. “The mismeasurement of quality and impact”. In Gaming the metrics: misconduct and manipulation in academic research, edited by Mario Biagioli and Alexandra Lippman, 66-75. Cambridge, The Mit press. https://doi.org/10.7551/mitpress/11087.003.0006. DOI: https://doi.org/10.7551/mitpress/11087.003.0006