Are There Drawbacks to Using GenAI for Research?

As I mentioned in my last post, there are many generative artificial intelligence (GenAI) tools that researchers and special librarians can use to conduct literature searching.

These tools can be beneficial for individuals new to a field and helpful when users want to visualize published research in new ways. However, there are also potential drawbacks to consider before signing off on GenAI as a reliable research assistant.

Researchers must take responsibility for understanding how GenAI works and closely review literature search results before relying on them. Let’s take a look at three concerns for researchers and special librarians to consider when leveraging or recommending GenAI platforms.

Potential Drawback #1: The Black Box Problem

One issue with research identified by GenAI tools is our lack of understanding of how the literature is selected. Just like with other GenAI outputs, the process of how GenAI selects literature is invisible. This is often referred to as the black box problem because we can’t “see” what’s going on behind the scenes; we only see the output delivered to us by the AI tools.

If the process for identifying literature is hidden from users, certain voices may be amplified over others without a clear understanding of why the GenAI tool is selecting certain literature. Additionally, the same literature may be referenced repeatedly instead of new content being highlighted. Asking GenAI to tell you the seminal works in a field would logically lead to many of the same sources being selected—but it may not highlight new research that could become seminal literature in the future.

Potential Drawback #2: Sloppy Summaries

Another concern with using GenAI tools to conduct research is that the tools will often summarize content. Individuals may be tempted to accept the summaries without verifying their accuracy, but AI-generated summaries are not the same as abstracts written by the authors. The AI summaries may be outright false or use language that is not precise, creating a different impression of the article than would be understood by someone who read the full piece.

Potential Drawback #3: The Relevance Spectrum

Further, GenAI tools may identify literature as relevant to a topic even though it is not actually well-aligned. Researchers will need to review AI-identified literature thoroughly to ensure its applicability to the research topic. In some cases, the identified literature may be tangentially related or discuss the research topic generally, but not specifically state what is understood by the researcher.

The Researcher’s Responsibility

While GenAI has many valuable uses, researchers should always be careful and diligently review any literature they locate with the help of AI. They should read the entire article—not just the abstract—and make sure they understand the authors’ argument. However, GenAI introduces new ways of locating resources that make it easier to skip labor-intensive close reading, potentially encouraging researchers to develop literature reviews without the same level of understanding they would have previously gained.

Therefore, as you are working with library users and other stakeholders in your job, I encourage you to be clear about how GenAI can be beneficial—and surface the issues it has the potential to create. Teaching users how GenAI works (i.e. large language models, predictive text) can help make these issues more understandable. When individuals better understand how GenAI works, they are more likely to be able to respond with appropriate actions and critical thinking about GenAI-created outputs.