Know Your Citations Better with Citation Context Analysis
Beyond simple counts of citations received by a paper, a novel method for examining impact of a piece of scholarly work—citation context analysis is able to “…help scholars explore and describe—at a specific and detailed level—how important ideas are used by and spread from a source text to subsequent citing works.” (Anderson and Lemken, 2020)
Several tools are now offering citation context analysis to give insights on how your paper is being cited by analyzing the citation statements. These tools are mining data from the citation statements to examine the citation intent of the authors through sentiment analysis or trained machine learning models. Analyzed data can be used to classify the citation type or category to indicate how your paper has made an impact on the cited paper.
As of the date the author wrote this post, there are two major systems that provide citation context analysis, namly Semantic Scholar and Scite.
Semantic Scholar
Semantic Scholar is a free, AI-powered research tool for scientific literature. While other traditional tools are showing the total citation number only and treat every citations equally, Semantic Scholar is able to provide you more context on your citations:
Citations are classified into three major categories – Background Citations, Methods Citations, and Results Citations.
Highly Influential Citations identifies important citations in the literature, in other words, it shows the number of citing publications that the cited publication has a significant impact on. As shown in the above example, this paper received 1,051 highly influential citations out of the 18,493 citations received in total. Influential citations are determined utilizing a machine-learning model analyzing a number of factors including the number of citations to a publication, and the surrounding context for each. More details of the adopted approach by Semantic Scholar can be found in “Identifying Meaningful Citations”.
Simply create a personal account on Semantic Scholar and you will be able to search its indexed publications on the platform and explore your scholarly work influence via citation context analysis. The platform has currently indexed over 205 million papers.
The below video briefly introduce you the citation categories used in Semantic Scholar:
Scite
Scite is another platform that uses a deep learning model that is well trained to identify three categories of citation statements: those that provide contrasting or supporting evidence for the cited work, and others, which mention the cited study without providing evidence for its validity.
Under the HKU Libraries subscription of scite, HKU authors can further create customized dashboards to analyze in which part of the citing papers your work is being cited. You may read more on the features of the citation context analysis provided by scite in another blog post: “Scite Dashboard: Analysis of HKU Publication Citations”.
Scholarcy
Unlike Semantic Scholar and Scite, Scholarcy is an AI-powered article summarizer that reads the references in your selected research articles, reports and book chapters and analyze the citation types and context. In other words, when Semantic Scholar and Scite are providing insights on how a target paper is being cited, Scholarcy does the reverse – how the target paper is citing others.
The main feature of Scholarcy is to read your selected or uploaded paper and break them down into bite-sized sections that summarize important information of the paper. Analysis on references used in the target paper is only one component in its analysis. Below shows a few example context that are extracted in the summary:
- Key Concepts
- Tables
- Figures
- Methods
- Results
- Limitations
- Funding information
- Ethics section
- References
These information can be viewed online on Scholarcy or exported in a flashcard.
Although some advanced features are only available behind a paywall, you could still make use of its flashcard creation feature for small-medium sized documents and installation of its browser extension on Chrome or Edge free-of-charge.
Scholarcy Flashboard
You may upload your selected paper or enter the URL if it is in open access on this page and generate the summary flashboard.
It will take a while for the system to read your uploaded paper. Results will show on the right panel once the flashcard has been successfully generated.
Chrome and Edge users may also wish to download and install the Scholarcy plugin at Chrome Store or Edge Store, however, we tested its performance and the Scholarcy citation context analysis works best by uploading the pdf file on the flashcard generator page.
Scholarcy analyzes and classifies references cited within the full-text article. The results can be viewed under the “Comparative analysis” section of the flashcard. There are five main citation types:
- Method details and prior work
- Builds on previous research
- Differs from previous work
- Confirmation of earlier findings
- Counterpoint to earlier claims
Examples:
From the examples, we can see that the extracted in-text references are categorized and it also shows the summarized section where the text appears, such as results, discussion, method details, study design, etc. “Differs from previous work” and “Builds on previous work” are statements that tell how the cited work or methods be used in the uploaded paper, while “Confirmation of earlier findings” and “Counterpoint to earlier claims” are telling the supporting or contrasting evidence to the cited reference within the paper.
Web of Science also announced that a new feature “enriched cited reference classified by use” is added in its new release in July 2021. This function is similar to what Scholarcy offers – classifying in-text mentions to indicate the reason which author may have cited the reference for. Web of Science will provide five categories in their classification – background, basis, discussion, support and differ. For more details, please read the release notes (on 22 July 2021) and another update (on 18 February 2022) posted by Web of Science.
Summary
Overall, due to the difference in nature of citation context analysis provided by the above introduced tools, Semantic Scholars and Scite are recommended for use in evaluating the impact of paper written by yourself or a particular author, and Scholarcy, as well as Web of Science soon, are suggested for use in analyzing papers of your interest during your literature review.
The below table summarizes the information of these tools:
| Name of tool | Size of collection | Citation types | Additional notes | Reference |
| Semantic Scholar | >205 million publications | 1. Cites Background 2. Cites Method 3. Cites Results | Identifies Highly Influential Citations | Details |
| Scite | 29 million full-text articles & 1 billion citation statements | 1. Mentioning cite 2. Supporting cite 3. Contrasting cite | 1. Detects retracted and withdrawn papers 2. Allows citation statement searches | Details |
| Scholarcy | N/A | 1. Method details and prior work 2. Builds on previous research 3. Differs from previous work 4. Confirmation of earlier findings 5. Counterpoint to earlier claims | 1. Analyze references used in a paper instead of cited references 2. No indexed context, extractions from uploaded contents | FAQ |
| Web of Science | >171 million records & 1.9 billion cited reference | 1. Background 2. Basis 3. Discussion 4. Support 5. Differ | 1. Analyze references used in a paper instead of cited references 2. In a pilot programme | Details |
Got any comments and questions? Please let us know your feedback or get in touch with us via scholarlycomm@hku.hk.
Reference:
Anderson, M., & Lemken, R. (2020). Citation Context Analysis as a Method for Conducting Rigorous and Impactful Literature Reviews. Organizational Research Methods. doi: 10.1177/1094428120969905.
