Measuring Research Impact

About Journal Metrics

Authors choose journals to submit their manuscript by evaluating the reputation of the journal, with several metrics as mentioned below.

1. Impact Factor: The impact factor (IF) is a measure of the frequency with which the average article in a journal has been cited for a particular year. It is also used to measure the ranking of a journal by calculating the times its articles are cited.
2. 5 Year Impact Factor: 5-year journal Impact Factor is the average number of times articles from the journal published in the past five years have been cited.
3. Eigen Factor: Eigen factor scores are intended to give a measure of how likely a journal is to be used and are thought to reflect how frequently an average researcher would access content from that journal.
4. SNIP (Source-normalized Impact per Paper): Source Normalized Impact per Paper (SNIP) measures contextual citation impact by weighting citations based on the total number of citations in a subject field.
5. SJR (Scimago Journal & Country Rank): The SCImago Journal & Country Rank is a portal that includes the journals and country scientific indicators developed from the information contained in the Scopus database.
6. Immediacy Index: The immediacy Index is the average number of times an article is cited in its published year.

Web of Science

The following journal impact metrics are used to measure impact factors in Web of Science.

1. Impact Factor: Impact Factor was developed in the 1960s and is commonly used to measure a journal's quality. It can be defined as "the average number of times articles cited from the journal published over a defined time." It can be calculated as No. of citations in the current year to an article published in the Journal for previous n years, divided by total no. of articles published in the Journal for last n years. The impact factor is available in “Journal Citation Report (JCR)”. The JCR provides quantitative tools for ranking, evaluating, categorizing, and comparing journals. It usually published Journal Impact Factor (JIF) report annually.
   
   How Impact factor is calculated: The calculation is based on a two-year period and involves dividing the number of times articles were cited by the number of articles that are citable.
    
   Calculation of 2020 IF of a journal: A = the number of times articles published in 2018 and 2019 were cited by indexed journals during 2020.
   B = the total number of "citable items" published in 2018 and 2019.
   A ÷ B = 2020 impact factor
2. 5 Year Impact Factor: The 5-Year Journal Impact Factor represents the average number of citations articles from the journal received in the past five years according to the JCR year.
   Broad Perspective - While offering a broader perspective on citation data, the 5-Year Journal Impact Factor sacrifices granularity due to reduced detail.
    
   Calculation: The number of citations in the JCR year ÷ total number of articles published in the 5 previous years = 5-year Journal Impact Factor
   Let's say Journal X published 100 articles in the past five years. During that time, these articles received a total of 500 citations. To calculate the 5-Year Impact Factor:
   Total citations = 500
   Total articles published = 100
   5-Year Impact Factor = Total citations ÷ Total articles published = 500 ÷ 100 = 5
   So, the 5-Year Impact Factor for Journal X is 5.
3. Immediacy Index: The immediacy Index is the average number of times an article is cited in its published year. It indicates how quickly articles in a journal are cited and is calculated by dividing the number of citations to articles published each year by the number of articles published in that year.
    
   Calculation: Cites in 2022 to the Articles published in 2022 = 6000
   No. of Articles published in 2022 = 400
   JII = Cites to Articles published ÷ No. of Articles published = 6000 ÷ 400 = 15
4. Eigen Factor: The 5-Year Journal Impact Factor is calculated as the number of citations received by articles published in a journal over the past 5 years, weighted by the influence of the citing journals. This metric allows for comparison across different subject fields and is accessible through the JCR or the website http://eigenfactor.org/. Similar to Google's PageRank algorithm, it leverages the entire citation network to evaluate the significance of each journal.
    
   Calculation: For example, if you were interested in finding the Eigenfactor of the journal "Nature," you could search for it on http://eigenfactor.org/ or in the Journal Citation Reports database. Once you locate the journal, you would view its Eigenfactor value and interpret its significance within the scientific community.
5. Essential Science Indicator: The ESI database offers insights into emerging science trends and influential entities such as individuals, institutions, papers, journals, and countries in research.
   ESI, integrated into the Web of Science Core Collection, ranks top-performing research across 22 fields based on publication and citation performance from over 11,000 journals worldwide.
   Updated bimonthly, ESI draws data from sources like SCIE and SSCI, providing timely and accurate information on scientific achievements and trends.
    
   Calculation: Let's say Author A has published 10 articles in neuroscience journals, with those articles collectively receiving 100 citations. Author B has published 5 articles in the same journals, with 150 citations in total.
   While Author B has fewer publications, their articles have received more citations on average, indicating higher impact.
   Therefore, Author B may be ranked higher than Author A in ESI's rankings for neuroscience.

Scopus

The following journal impact metrics are used to measure impact factors in Scopus.

1. SJR (SCImago Journal Rank): The SCImago Journal Rank (SJR) is a metric used to assess the scientific prestige and influence of scholarly journals. It measures the average number of citations received by articles published in a journal over a specific period, while also considering the quality of the journals where those citations come from.
    
   Calculation: Let's consider Journal X, which published 100 articles in the past year. During that time, these articles received a total of 500 citations. However, not all citations are weighted equally. For example, citations from prestigious journals are given more weight than those from less influential sources.
   Assuming that 400 of the citations come from high-ranking journals and 100 from lower-ranking journals, the weighted sum of citations is calculated accordingly. Then, this sum is divided by the total number of documents in the journal's reference list (100 articles in this case) to obtain the SJR score.
   Let's say the weighted sum of citations is 800.
   SJR = Weighted sum of citations ÷ Total number of documents = 800 ÷ 100 = 8
   So, the SJR score for Journal X is 8.
2. SNIP (Source Normalized Impact Per Paper): The Source Normalized Impact per Paper (SNIP) is a bibliometric indicator used to assess the citation impact of scholarly journals. It measures the average citation impact of papers in a journal, taking into account the citation potential of the subject field in which the journal operates.
    
   Calculation: Let's consider Journal Z, which belongs to the field of molecular biology.
   The average citation count for papers published in Journal Z over a specific period is calculated, let's say it's 10 citations per paper.
   Additionally, the average citation rate for papers in the field of molecular biology as a whole is determined to be 5 citations per paper.
   The SNIP score for Journal Z is then calculated by dividing the average citation count of Journal Z (10) by the average citation rate in the field of molecular biology (5).
   SNIP = Average citation count of Journal Z ÷ Average citation rate in the field = 10 ÷ 5 = 2
   So, the SNIP score for Journal Z is 2.
3. CiteScore: CiteScore is a metric used to evaluate the citation impact of scholarly journals. It measures the average number of citations received per document published in a journal over a specific period, typically one year.
    
   Calculation: Let's consider Journal Y, which published 100 documents (articles, reviews, etc.) in the year 2023. During that same year, the documents published by Journal Y received a total of 500 citations.
   The CiteScore for Journal Y in 2023 is calculated by dividing the total number of citations (500) by the total number of documents published (100).
   CiteScore = Total citations ÷ Total documents = 500 ÷ 100 = 5
   So, the CiteScore for Journal Y in 2023 is 5.

Google Scholar

The following journal impact metric is used to measure impact factors in Google Scholar.

1. Google Scholar Metrics: Google Scholar Metrics is a freely accessible tool provided by Google that allows users to explore the citation metrics of scholarly journals and publications. It provides rankings of journals and articles based on their citation metrics, offering insights into their influence and impact within the academic community.
   It provides researchers, academics, and institutions with valuable information about the impact and visibility of scholarly publications and journals, helping them make informed decisions about where to publish and which research to follow.
    
   Calculation: Let's consider Journal X, which has an h5-index of 20 and an h5-median of 50.
   This means that the journal has published at least 20 articles in the last 5 years, each of which has been cited at least 20 times.
   Additionally, the median number of citations for these articles is 50.

About Article Level Metrics

Article Level Metrics (ALMs) refer to a set of metrics that measure the overall performance and reach of a single article. These metrics offer a more comprehensive view of the article's impact compared to traditional citation counts. Here are some common Article Level Metrics:

1. Citations: The number of times the article has been cited by other articles. This is the most traditional metric of an article's impact.
2. Views: The number of times the article has been viewed or downloaded. This can indicate the level of interest in the article.
3. Social Media Mentions: The number of times the article has been shared or mentioned on social media platforms. This can demonstrate the article's reach and engagement with the public.
4. Bookmarks: The number of times the article has been bookmarked or saved for later reading. This can indicate the perceived value of the article to readers.
5. Comments: The number of comments or discussions generated by the article. This can show the level of engagement and interest in the article's content.

Resources to Measure

The following resources provide article-level metrics:

1. Altmetric: It tracks and analyzes online activity such as mainstream media, social media, and publisher download data for mentions of scholarly literature. This data is used to calculate an Altmetric score, which is a measure of the quality and quantity of attention that an article has received.
2. PLOS Article-Level Metrics: PLOS provides article-level metrics, such as the number of views and downloads, for articles published in its journals.
3. Bookmetrix: Bookmetrix is an analytics platform for books, providing authors, publishers, and institutions with insights into the reach and impact of their publications.
4. Dimensions: Dimensions is a discovery and analytics platform that provides article-level metrics, including the number of citations, Relative Citation Ratio, and altmetrics, for more than 90 million publications.
5. CINAHL: This database indexes nursing, consumer health, and allied health journals, trade publications, dissertations, and books. CINAHL provides article-level metrics, such as the number of abstract views or mentions on Twitter, from Plum Analytics.
6. Scopus (Plumx Metrics): This database offering access to scholarly literature and provides PlumX Metrics for assessing research impact through various indicators like citations, social media mentions, and media coverage.
7. Web of Science: The Web of Science provides the number of citations per article.
8. Google Scholar: it Searches scholarly literature available on the web and provides the number of citations per article.
9. PubMed: PubMed is a database of more than 30 million citations for biomedical literature from MEDLINE, life science journals, and online books.
10. Mendeley: Mendeley provides information on readership (included in Scopus PlumX Metrics).

Tools

Altmetrics Tools, as discussed earlier in this report, utilize Article-Level Metrics (ALMs) that amalgamate data from diverse sources, including traditional (e.g., times cited) and modern (e.g., tweets). Various tools have arisen to capture and present these alternative metrics, known as altmetrics. This appendix lists several prominent ones.

1. Altmetric: Backed by Digital Science, Macmillan’s technology incubator, Altmetric has been adopted by Springer, Nature Publishing Group, Scopus, and BioMed Central, among others. Altmetric tracks social media sites, newspapers, and magazines for any mentions of hundreds of thousands of scholarly articles. Altmetric then creates a score for each article. This is a quantitative measure of the quality and quantity of attention that a scholarly article has received. It is based on three main factors: the number of individual mentioning a paper, where the mentions occurred (e.g., a newspaper, a tweet), and how often the author of each mention talks about scholarly articles. Altmetric is a for-profit entity.
2. ImpactStory: ImpactStory is an open-source altmetric tool, its code available freely for anyone to use. ImpactStory (formerly Total Impact) draws from a variety of social and scholarly data sources, including Facebook, Twitter, CiteULike, Delicious, PubMed, Scopus, CrossRef, scienceseeker, Mendeley, Wikipedia, slideshare, Dryad, and figshare. ImpactStory normalizes metrics based on a sample of article published the same year; altmetrics are reported in both raw scores and percentiles compared to other articles. ImpactStory offers a free widget to embed metrics on any web page. It is a nonprofit entity.
3. Plum Analytics: A Seattle-based technology startup, Plum Analytics aims to track metrics for nearly two dozen types of outputs, including journal articles, book chapters, datasets, presentations and source code. Its focus product provides custom reports intended to quantify departmental productivity, support grant proposals, and address other impact related questions. PlumX is marketed to universities and other research institutions a way to track researchers’ productivity.
4. PLOS: PLOS has developed and released a Ruby on Rails application that stores and reports user configurable performance data on research articles. The open source utility can be customized to track ALMs for specific articles, and to include additional data sources for deriving the metrics.

About Author Level Metrics

Author impact is typically assessed by the number of citations their work garners. Tracking an author's complete list of citations poses a challenge.
Establishing an author's profile with a persistent identifier like ORCID ID or Researcher ID can address this issue.

Author metrics serve to gauge the frequency of citations to an author's work. They illustrate the reach and impact of a researcher's contributions.
These metrics are crucial for grant applications, tenure evaluations, promotion decisions, and performance reviews.

H-Index

The H-Index is a widely-used metric in academia that quantifies both the productivity and impact of a researcher's publications.

1. What is H-Index: The h-index is a widely recognized metric in academia that aims to quantify both the productivity and impact of a researcher's body of work. It provides a balance between the number of publications and the number of citations those publications have received, offering insight into the overall influence of a researcher's contributions within their field.
    
   Calculation: Let's consider Dr. Taylor, a researcher who has published several papers. Among these papers, their top 7 publications have received the following citation counts: 12, 10, 8, 6, 5, 4, and 2.
   Dr. Taylor's h-index would be 5, indicating that they have at least 5 papers that have each been cited at least 5 times.

G-Index

The G-Index is a metric that measures the highest number of papers an author has published that together have received at least the same number of citations as the number of papers.

1. What is G-Index: The g-index is a valuable metric in academia that offers an alternative perspective to assess the impact of a researcher's publications. It aims to capture both the quantity and impact of an author's work.
    
   Calculation: Imagine a researcher, Dr. Smith, who has published 5 papers. These papers have received the following number of citations: 10, 8, 6, 4, and 2.
   To calculate Dr. Smith's g-index, we arrange their papers in descending order of citations and find the point where the cumulative number of citations matches the number of papers published.
   In this case, Dr. Smith's top 3 papers have received a total of 24 citations.
   Therefore, Dr. Smith's g-index is 3.

i10-Index

The i10-Index is a metric in Google Scholar that represents the number of publications by an author that have received at least 10 citations each.

1. What is i10-Index: In the realm of scholarly impact metrics, the i10-index, pioneered by Google Scholar, shines a spotlight on the quality of an author's work by counting the number of their publications that have garnered a minimum of 10 citations each, providing a succinct measure of their influence in academia.
    
   Calculation: Let's consider Dr. Robert, a researcher in environmental science, who has published 8 papers.
   Among these, 5 have received at least 10 citations each.
   Therefore, Dr. Robert's i10-index is 5.