Halt the h-index
Using the h-index in research evaluation? Rather not. But why not, actually? Why is using this indicator so problematic? And what are the alternatives anyway? Our authors walk you through an infographic that addresses these questions and aims to highlight some key issues in this discussion.
Sometimes, bringing home a message requires a more visual approach. That’s why recently, we teamed up with a graphic designer to create an infographic on the h-index – or rather, on the reasons why not to use the h-index.
In our experience with stakeholders in research evaluation, debates about the usefulness of the h-index keep popping up. This happens even in contexts that are more welcoming towards responsible research assessment. Of course, the h-index is well-known, as are its downsides. Still, the various issues around it do not yet seem to be common knowledge. At the same time, current developments in research evaluation propose more holistic approaches. Examples include the evaluative inquiry developed at our own centre as well as approaches to evaluate academic institutions in context. Scrutinizing the creation of indicators itself, better contextualization has been called for, demanding to derive them out “in the wild” and not in isolation.
Moving towards more comprehensive research assessment approaches that consider research in all its variants is supported by the larger community of research evaluators as well, making a compelling case to move away from single-indicator thinking.
Still, there is opposition to reconsidering the use of metrics. When first introducing the infographic on Twitter, this evoked responses questioning misuse of the h-index in practice, disparaging more qualitative assessments, or simply shrugging off responsibility for taking action due to a perceived lack of alternatives. This shows there is indeed a need for taking another look at the h-index.
The h-index and researcher careers
To begin with, the h-index only increases as time passes. This time-dependence means that the h-index favors more senior researchers over their younger colleagues. This becomes clearly apparent in the following (fictitious) case, which compares a researcher at a more advanced career stage to an early-career researcher. As can be seen in Figure 1, the researcher at the more advanced career stage has already published a substantial number of publications.
Figure 1. Publications of a researcher at a more advanced career stage. Publications can be seen on the x-axis and are represented on the right as well.
Over time, the publications depicted attract more and more citations (see the y-axis in Figure 2).
Figure 2. Citations for publications of a researcher at a more advanced career stage. On the right: colored representations of citations received as they occur in other publications.
When ordering a researcher’s list of publications by the (decreasing) number of citations received by each of them, you will get a distribution as in Figure 3. The value of the h-index equals the number of publications (N) that have received N or more citations. In our example, this results in an h-index of five.
Figure 3. Calculating the h-index for a researcher at a more advanced career stage
The situation is less favorable for the early-career researcher. This is because the h-index does not account for academic age: researchers who have been around for a shorter amount of time fare worse with this metric. Hence, while the h-index provides an indication of a researcher’s career stage, it does not enable fair comparisons of researchers that are at different career stages.
Figure 4. Calculating the h-index for an early-career researcher.
The h-Index and field differences
Another problem of the h-index is that it does not account for differences in publication and citation practices between and even within fields. A closer look at another hypothetical case may exemplify this issue. We are taking a look at two different fields of science: One with a low (Figure 5) and another one with a high publication and citation density (Figure 6). The researcher in the field with a low publication and citation density has an h-index of two, while the h-index equals five for the researcher in the field with a high publication and citation density. The difference in the h-indices of the two researchers does not reflect a difference in scientific performance. Instead, it simply results from differences in publication and citation practices between the fields in which the two researchers are active.
Figure 5. Calculating the h-index for a researcher active in a field with a low publication and citation density.
Figure 6. Calculating the h-index for a researcher active in a field with a high publication and citation density.
Three problems of the h-index
The application of the h-index in research evaluation requires a critical assessment as well. Three problematic cases can be identified in particular (Figure 7). The first one, in which the h-index facilitates unfair comparisons of researchers, has already been discussed.
Figure 7. Three problems of the h-index
The h-index favors researchers who have been active for a longer period of time. It also favors researchers from fields with a higher publication and citation density (Figure 8).
Figure 8. Problem 1: Unfair comparisons.
Another problem of the h-index is that merely publishing a lot becomes a ‘virtue’ in its own right. This means that researchers benefit from placing their name on as many publications as possible. This may lead to undesirable authorship behavior (Figure 9). Likewise, the strong emphasis of the h-index on citations may cause questionable citation practices.
Figure 9. Problem 2: Bad publishing and referencing behavior.
There is yet another issue. By placing so much emphasis on publication output, the h-index narrows research evaluation down to just one type of academic activity. This runs counter to efforts towards a more responsible evaluation system that accounts for other types of academic activity as well: Leadership, vision, collaboration, teaching, clinical skills, or contributions to a research group or an institution (Figure 10).
Figure 10. Problem 3: Invisibility of other academic activities.
Relying on just a single indicator is shortsighted and is likely to have considerable narrowing effects. Topical diversity may suffer since, as shown above, some fields of science are favored over others. Moreover, a healthy research system requires diverse types of talents and leaders, which means that researchers need to be evaluated in a way that does justice to the broad diversity in academic activities, rather than only on the basis of measures of publication output and citation impact.
Halt the h-index and apply alternative approaches for evaluating researchers
How then do we “halt the h-index”? We would like to emphasize a threefold approach when it comes to evaluating academics (Figure 11). The starting point is a qualitative assessment by one’s peers and colleagues. Adding to that, indicators applied in the assessment should be both transparent and flexible. Transparency means that the calculation of an indicator is clearly documented and that the data underlying an indicator are disclosed and made accessible. Thus, “black box” indicators are not acceptable. Regarding flexibility, the assessment should not apply a ‘one-size-fits-all’ approach. Instead, it should adapt to the type of work done by a researcher and the specific context of the organization or the scientific field in which the researcher is active. Lastly, we believe that written justifications, e.g. narratives or personal statements, should become an important component in evaluations, including in the context of promotions or applications for grants. Such statements could for instance highlight the main accomplishments of researchers and their future plans.
Figure 11. Evaluating academics in a responsible way.
In conclusion, evaluation of individual researchers should be seen as a qualitative process carried out by peers and colleagues. This process can be supported by quantitative indicators and qualitative forms of information, which together cover the various activities and roles of researchers. The use of a single unrepresentative, and in many cases even unfair, indicator based on publication and citation counts is not acceptable.
This conclusion very much aligns with calls for broader approaches towards recognition and rewards in academia, and was also specified in the Leiden Manifesto:
“Reading and judging a researcher's work is much more appropriate than relying on one number. Even when comparing large numbers of researchers, an approach that considers more information about an individual's expertise, experience, activities and influence is best.” (Principle 7 in the Leiden Manifesto)
What remains to be done? Take a look at the infographic and feel free to share and use it! All the visuals used in this blog post combined in one infographic, ready for download and shared under a CC-BY license, can be found on Zenodo.
Acknowledgements
We would like to thank Robert van Sluis, the designer who produced the images shown in this blog post and the final infographic.
Many thanks also to Jonathan Dudek and Zohreh Zahedi for all their help in preparing this blog post.
Photo by Nick Wright on Unsplash
4 Comments
Доброго времени суток товарищи<a href=https://2612.by/>!</a>
<a href=https://2612.by/><img src="https://i116.fastpic.org/big/2021/1219/21/58842228f6ccf3c1f8c35b18a1109a21.jpg"></a>
Заправка картриджей Минск – это скрупулезная и ответственная работа, так как от нее зависит рабочий процесс всей компании или распечатка важного документа. Малейшая ошибка или недоделка приведет к перебоям в работе заказчика, и соответственно к потере нашей репутации и прибыли. Для нас на первом месте стоит репутация, которую мы заработали нелегкими и долгими годами труда. Поэтому выполняя очередной заказ на заправку принтера или ксерокса, мы думаем о клиенте и с полной ответственностью можем гарантировать как качество, так и быстрое исполнение работы.Заявку на вызов Вы можете оставить как на сайте, так и по телефону. Так же в Online чате сбоку. Наш оператор обговорит удобное для Вас время, и наши специалисты прибудут точно в срок. Качество наших работ всегда на высшем уровне, так как у нас работает лучшая команда профессионалов!Рады предложить нашим клиентам:Заправка картриджей с выездом на дом и офис.Ремонт и восстоновление картриджей.Прошивка любого лазерного принтера.Настройка принтера (wi-fi, установка драйверов на компьютер).Замена термопленки и роликов захвата бумаги.Купить картридж к любому лазерному картриджу.Для заказа мастера просто позвоните по указанным телефонам или оставьте завку, выбрав свою модель!
Наша фирма занимается свыше 10 лет ремонтом и обслуживанием оргтехники в городе Минске.Основные направления и виды нашей деятельности:
1)<a href=https://2612.by/>заправка картриджей минск</a>
2)<a href=https://2612.by/>hp заправка картриджей</a>
3)<a href=https://2612.by/>заправка картриджа canon</a>
4)<a href=https://2612.by/>brother принтер картридж</a>
5)<a href=https://2612.by/>заправка xerox phaser</a>
6)<a href=https://2612.by/>ремонт принтеров минск</a>
Мы будем рады Вас видеть у нас на интернет ресурсе <a href=https://2612.by/>НАЖМИТЕ ССЫЛКУ</a>
Всегда рады помочь Вам!С уважением,КОПИМЕДИАГРУПП
<a href=https://2612.by/>чернила inktec для canon pixma</a>
<a href=https://2612.by/>тефлоновый вал konica minolta</a>
<a href=https://2612.by/>чернила эпсон купить</a>
<a href=https://2612.by/>oki 44574302</a>
<a href=https://2612.by/>тонер xerox 700</a>
<a href=https://2612.by/>купить заправку для картриджей brother</a>
<a href=https://2612.by/>лазерный картридж hp ce285a черный</a>
<a href=https://2612.by/>oki hentai</a>
<a href=https://2612.by/>canon imageprograf tm 300 чернила</a>
<a href=https://2612.by/>чернила для принтера epson набор</a>
<a href=https://2612.by/>купить оригинальные чернила canon</a>
<a href=https://2612.by/>чипы xerox</a>
<a href=https://2612.by/>brother 130c картридж</a>
<a href=https://2612.by/>xerox workcentre 3045 чип</a>
<a href=https://2612.by/>kyocera km 1500 картридж</a>
<a href=https://2612.by/>oki c712</a>
<a href=https://2612.by/>набор чернил для принтера hp</a>
<a href=https://2612.by/>принтер brother hl 1202r картридж</a>
<a href=https://2612.by/>чип hp designjet 500</a>
<a href=https://2612.by/>kyocera пишет неоригинальный картридж с тонером</a>
<a href=https://2612.by/>kyocera ecosys m2235 картридж</a>
<a href=https://2612.by/>тонер kyocera tk 435</a>
<a href=https://2612.by/>canon pixma ix6840 чернила</a>
<a href=https://2612.by/>чернила hp x4e40ae</a>
<a href=https://2612.by/>oki mb472</a>
<a href=https://2612.by/>купить чернила для принтера canon pixma g1411</a>
<a href=https://2612.by/>замена ролика подачи бумаги samsung 4200</a>
<a href=https://2612.by/>kyocera 3100 тонер</a>
<a href=https://2612.by/>заправка картриджа canon 725</a>
<a href=https://2612.by/>чернила для принтера epson инксистем</a>
<a href=https://2612.by/>фоторецептор hp</a>
<a href=https://2612.by/>hp m1132 замена термопленки</a>
<a href=https://2612.by/>canon i sensys mf3010 заправка картриджа</a>
<a href=https://2612.by/>hp smart tank чернила</a>
<a href=https://2612.by/>xerox 3020 тонер</a>
<a href=https://2612.by/>oki doki</a>
<a href=https://2612.by/>xerox 6000 чипы</a>
<a href=https://2612.by/>canon 5140 чернила</a>
<a href=https://2612.by/>чернила hp 510</a>
<a href=https://2612.by/>oki c9655</a>
<a href=https://2612.by/>термопленка hp lj 1022</a>
<a href=https://2612.by/>тонер картриджи для мфу xerox</a>
<a href=https://2612.by/>kyocera мфу неоригинальный картридж</a>
<a href=https://2612.by/>kyocera p2335d картридж</a>
<a href=https://2612.by/>чернила hp ink tank</a>
<a href=https://2612.by/>panasonic mb2000 тонер</a>
<a href=https://2612.by/>canon pixma g3411 чернила купить</a>
<a href=https://2612.by/>brother dcp 7055r сброс счетчика тонера</a>
<a href=https://2612.by/>фотобарабан hp w1104a</a>
<a href=https://2612.by/>тонер xerox wc 5325</a>
Доброго времени суток господа<a href=https://2612.by/>!</a>
<a href=https://2612.by/><img src="https://i116.fastpic.org/big/2021/1219/21/58842228f6ccf3c1f8c35b18a1109a21.jpg"></a>
Помимо ремонта принтеров мы предлагаем Вам профессиональную заправку картриджей.Наша компания предлагает такую услугу как ремонт, заправка, прошивка, и сервисное обслуживание принтеров и другой оргтехники в офисах и на дому всех популярных моделей Canon, Xerox, Samsung, HP laserjet, Kyocera, Ricoh, Brother, PANTUM.Заправка черно-белых картриджей, заправка цветных картриджей, восстановление, бесплатная диагностика, замена фотобарабана, бесплатный выезда мастера на дом.Домашний принтер отказывается печатать? Работа встала из-за поломки офисного МФУ? Не устраивает качество печати или постоянно застревающая бумага? Наши специалисты помогут быстро и эффективно настроить работу вашей техники.Мы предлагаем профессиональный ремонт принтеров в Минске. Наши опытные специалисты осуществляют обслуживание любых моделей всех известных производителей. Не первый год предоставляя услуги жителям столицы, мы гарантируем квалифицированную помощь и использование качественных комплектующих.ремонт принтеров.ПРИНТЕРНАЯ ПОМОЩЬ С ВЫЕЗДОМ НА ДОМ.Сотрудники нашей компании обладают многолетним опытом ремонта принтерного оборудования и высоким уровнем профессионализма. Кроме того, нашим клиентам доступна такая услуга, как выезд специалиста на дом или в офис.Опытный профессионал произведет обязательную диагностику принтера непосредственно на дому или в офисе вашей компании, при необходимости выполнит замену подвижных частей, настроит работу оборудования. Если же техника потребует более сложного вмешательства, мы самостоятельно заберем МФУ или принтер для последующего ремонта, а затем вернем его по истечении указанного срока.Мы осуществляем ремонт принтеров с использованием оригинальных и соответствующих техническим требованиям деталей. Наши специалисты имеют опыт работы с оборудованием различных марок, знают о типичных неисправностях конкретных моделей и способах устранения неполадок. Мы ценим свою репутацию, поэтому заботимся о качестве и долговечности используемых комплектующих.Звоните, и мы поможем вам вернуть принтерную технику в рабочее состояние. Не стоит отказывать себе в возможности быстро и эффективно восстановить работоспособность оборудования и продолжить продуктивную и плодотворную работу. Мы выезжаем на адрес клиента. Вы не успеете нам позвонить, как мы уже будет в пути.
Наша компания занимается свыше 10 лет ремонтом и обслуживанием оргтехники в городе Минске.Основные направления и виды нашей деятельности:
1)<a href=https://2612.by/>заправка картриджей минск</a>
2)<a href=https://2612.by/>hp заправка картриджей</a>
3)<a href=https://2612.by/>заправка картриджа canon</a>
4)<a href=https://2612.by/>brother принтер картридж</a>
5)<a href=https://2612.by/>заправка xerox phaser</a>
6)<a href=https://2612.by/>ремонт принтеров минск</a>
Мы будем рады Вас видеть у нас на интернет ресурсе <a href=https://2612.by/>НАЖМИТЕ ССЫЛКУ</a>
Всегда рады помочь Вам!С уважением,КОПИМЕДИАГРУПП
<a href=https://2612.by/>ремонт компьютера заправка картриджа</a>
<a href=https://2612.by/>kyocera 1500 картридж</a>
<a href=https://2612.by/>epson tx650 чернила</a>
<a href=https://2612.by/>чернила hp gt51 черный</a>
<a href=https://2612.by/>заправка картриджей для принтера минск</a>
<a href=https://2612.by/>установлен неизвестный тонер kyocera</a>
<a href=https://2612.by/>чернила для canon g3411</a>
<a href=https://2612.by/>картридж panasonic купить</a>
<a href=https://2612.by/>замена барабана картриджа</a>
<a href=https://2612.by/>oki mb461</a>
<a href=https://2612.by/>oki печать</a>
<a href=https://2612.by/>фотобарабан kyocera купить минск</a>
<a href=https://2612.by/>картридж canon 047 заправка</a>
<a href=https://2612.by/>cf230a заправка</a>
<a href=https://2612.by/>тонер картридж sharp</a>
<a href=https://2612.by/>тонер для принтера brother</a>
<a href=https://2612.by/>xerox versalink b7025 тонер</a>
<a href=https://2612.by/>бушинг магнитного вала hp купить</a>
<a href=https://2612.by/>тонер brother hl 2040</a>
<a href=https://2612.by/>тонер brother купить минск</a>
<a href=https://2612.by/>неоригинальный картридж с тонером kyocera что делать</a>
<a href=https://2612.by/>oki</a>
<a href=https://2612.by/>комплект чернил для epson l800</a>
<a href=https://2612.by/>картридж lexmark купить минск</a>
<a href=https://2612.by/>принтер kyocera закончился тонер</a>
<a href=https://2612.by/>canon pixma mg3640s чернила</a>
<a href=https://2612.by/>картридж kyocera tk 3060</a>
<a href=https://2612.by/>hp laserjet картридж цветной</a>
<a href=https://2612.by/>фотобарабан panasonic</a>
<a href=https://2612.by/>резиновый вал xerox cet</a>
<a href=https://2612.by/>hp color laserjet pro m282nw чип</a>
<a href=https://2612.by/>epson l386 чернила</a>
<a href=https://2612.by/>чернила для принтера epson l132</a>
<a href=https://2612.by/>чернила epson l366 купить</a>
<a href=https://2612.by/>картридж kyocera tk 1170</a>
<a href=https://2612.by/>заправка картриджа 435</a>
<a href=https://2612.by/>brother 1512r заправка картриджа</a>
<a href=https://2612.by/>brother hl 2132r тонер</a>
<a href=https://2612.by/>canon pixma ts5040 чернила</a>
<a href=https://2612.by/>ролик заряда kyocera</a>
<a href=https://2612.by/>купить фотобарабан kyocera</a>
<a href=https://2612.by/>чернила для принтеров canon ip7240</a>
<a href=https://2612.by/>oki b411dn</a>
<a href=https://2612.by/>oki b410</a>
<a href=https://2612.by/>hp laser mfp 135w чип</a>
<a href=https://2612.by/>kyocera 3060 картридж</a>
<a href=https://2612.by/>тонер картридж для принтера kyocera</a>
<a href=https://2612.by/>тонер картридж xerox wc</a>
<a href=https://2612.by/>пцр hp</a>
<a href=https://2612.by/>чернила для принтера epson px660</a>
@ Lutz Bornmann : Having field-specific comparisons would perhaps reduce the field difference bias, but from my perspective where we fund interdisciplinary research, this would prove to be quite difficult to do as researchers quite often tend to publish in different fileds.
Note that I am not a proponent of the h index. For example, some years ago, we developed beamplots that are used by Clarivate Analytics now as better alternative to the h index in the Web of Science (https://clarivate.com/blog/the-web-of-science-author-impact-beamplots-a-new-tool-for-responsible-research-evaluation/). In my opinion, the comment by Sarah, Ludo, and Thed is partly misleading and inconsistent. The first critical point in the comment (the possible advantage of senior scientists by the h index) has been targeted by Jorge Hirsch himself in 2005: the simple division of the h index by the number of years since publishing the first paper. The second critical point in the comment concerns the point that the h index does not account for field differences in citation impact (that are independent from quality issues). This point is counteracted by the authors of the comment themselves: they propose to adapt the choice of indicators and comparisons to the field-specific environment of the evaluation. This adaption also means in many cases that comparisons across fields are not done and indicators that are not field-normalized such as the h index can be used in principle. The authors recommend at the end of the comment not to use ‘black box’ indicators with data that are not openly available. In my opinion, the h index is an excellent example not being a ‘black box’ indicator (it is very simple and easy to understand) that can easily be used with open data. When I read the comment, I wondered why the authors did not mention the paper by Ludo and Nees (10.1002/asi.21678) in which they demonstrate the inconsistency of the h index in certain situations. This paper provides convincing evidence not to use the h index.
Add a comment