Researcher Connect

Latest news and trends on research support by HKU Libraries

Research Metrics Scholarly Communication Services

Capturing Impacts of Non-Traditional Research Outputs (NTROs)

Post Views: 41

— by Fanny Liu

Introduction to NTROs

Non-Traditional Research Outputs (NTROs) are research outputs beyond the traditional research outputs (academic journal articles, conference papers, academic books, and book chapters). They represent a diverse spectrum of creative and practice-based works, reflecting the innovative nature of research, particularly within the humanities and social sciences.

For example, in the ERA (Excellence in Research for Australia) 2018 by Australian Research Council (2020), NTROs have been submitted for review, including but not limited to:

  • original creative works
  • live performance of creative works
  • recorded/rendered creative works
  • curated or produced substantial public exhibitions and events
  • research reports for an external body
  • portfolio

Importance of considering NTROs

In Hong Kong, the upcoming RAE (Research Assessment Exercise) 2026 indicates the submission of NTROs cases as acceptable research outputs, with a definition as follows (University Grants Committee, 2024, p. 13):

“Other forms of output that was published or made publicly accessible or effectively shared within the profession, e.g. performance recording, video tape, computer software programme, architectural drawings, or any creative work that can be evaluated for merit and an assessment obtained.”

Non-traditional research outputs offer insights into applied research, particularly creative and practitioner-based research in the humanities and social sciences. For example, in the ERA 2018 by Australian Research Council (2020), a total of 4266 original creative works, 892 live performances, 901 recorded or rendered works, 626 curated or exhibited events, 3573 research reports for an external body, and 880 portfolios were submitted.

Also, the processes and outputs of interdisciplinary research may not necessarily follow traditional formats. To recognise and reward interdisciplinary research, the forthcoming REF (Research Excellence Framework) 2029 in the United Kingdom would remove the minimum 2* requirement for research underpinning impact case studies, with an aim to increase institutions’ confidence in impacts arising from research presented in a non-traditional output form (Research England et al., 2023, p. 12).

Capturing impact of NTROs

Capturing the impact of NTROs can be challenging because NTROs, unlike traditional academic outputs, lack a systematic infrastructure for impact measurement. To address this, Deakin University Library (2025) and the University of Melbourne Library (2025) provide guides on research metrics related to NTROs, suggesting several strategies as follows.

Combine Tools

Multiple tools can be used for a more comprehensive set of measures. As NTROs may not be cited as frequently as traditional research outputs, other custom metrics may be quoted. Search engines such as Google would be useful. Here are two examples on finding metrics on books:

The best practice is to keep a record of all the awards, prizes, commissions or royalties, book or box office sales, grants, reviews, invitation to perform/exhibit, media and news mentions, audience or visitor numbers, prestige of venues, and esteem of performers.

Provide Qualitative Analysis

Supplementing quantitative metrics, qualitative feedback, reviews, or measures should be provided to highlight the NTRO’s impact. Here are some examples on finding reviews:

The source of the data and the date it was generated should be documented to maintain transparency and relevance.

Write a Narrative

Metrics should be combined with a thoughtful narrative to present a clearer, more comprehensive picture of the NTRO’s influence. A compelling narrative contextualises the numbers by telling the story with:

  • Relevance: The extent to how closely an NTRO aligns with real-world needs, current practices, or ongoing policy development.
  • Innovation: The degree of originality and creativity in the NTRO, highlighting new developments and practical applications.
  • Engagement: The level of interaction of the NTRO with the intended audience or community.
  • Influence: The measurable effect of the NTRO on the field, community, or society.
  • Longevity: The lasting impact for the NTRO.

Conclusion

In summary, apart from traditional academic outputs, NTROs can be used to showcase the impacts of research, especially in the humanities and social sciences. By employing a combination of quantitative and qualitative measures, supported by a compelling narrative, researchers can effectively communicate the contributions of NTROs to the stakeholders.

Extended Readings

References

Australian Research Council. (2020). State of Australian University Research 2018–19: ERA National Report. https://dataportal.arc.gov.au/era/nationalreport/2018/

Deakin University Library. (2025). Research Metrics Toolkit. Retrieved 25 March 2025 from https://deakin.libguides.com/research-metrics/NTRO

Research England, Scottish Funding Council, Medr, Wales’ Commission for Tertiary Education and Research, & Department for the Economy, Northern Ireland. (2023). Research Excellence Framework 2029: Initial decisions and issues for further consultation. https://repository.jisc.ac.uk/9148/1/research-excellence-framework-2028-initial-decisions-report.pdf

The University of Melbourne Library. (2025). Research impact for Education: A self-help guide. Retrieved 25 March 2025 from https://unimelb.libguides.com/FOE-research-impact/non-standard

University Grants Committee. (2024). Research Assessment Exercise 2026 Guidance Notes. https://www.ugc.edu.hk/eng/ugc/activity/research/rae/2026/guidance_notes.html

Declaration of Generative AI use

I acknowledge the use of Generative AI tools in writing this post. I used:

  • GPT-4o to check for grammar mistakes and typos in the content I wrote.   

I declare that I reviewed and edited the contents as needed, and take full responsibility for the content of the post; And the information provided is complete and accurate.

Share
Tags: