In the previous series, we explored how AI is changing knowledge work. This new series takes a closer look at how AI is changing the experience of knowledge services, from discovery and access to institutional responsibilities and the ways organizations demonstrate their value.
We begin with something increasingly familiar: the chatbot.
Conversational interfaces are becoming a common way to interact with information systems. They make it easier to ask questions, explore ideas, and find information without having to understand the systems behind them. But as we become more comfortable asking chatbots for answers, how do we know whether those answers deserve our trust?
What Exactly Are We Talking To?
Chatbots have been around for decades, and their reputation is mixed. Anyone who has struggled with an automated customer service assistant will recognize the frustration of repeating a question, navigating scripted responses, or trying to reach someone who can actually help.
Early chatbots relied on predefined rules, keywords, and conversational pathways. Generative AI enabled more natural responses. But a fluent conversation does not necessarily mean the system understands the research problem or has access to reliable information.
This is where the underlying technology matters. Retrieval-augmented generation (RAG) connects a language model to external sources, allowing it to retrieve information before generating a response. Agentic systems can go further, using tools to search, analyze documents, and carry out sequences of tasks.
These approaches can overlap. The chatbot is increasingly the visible entrance to a much more complicated information environment. And that raises an interesting possibility: what if we could interrogate an entire collection through conversation, rather than searching for individual documents?
Why Chatbots Work Well (In Theory)
Two related developments help explain the appeal of conversational AI: the continuity of conversation and the ability to work with information in context. Conversation is a natural way to think. We ask questions, test ideas, make connections, and change direction as our understanding develops.
Unlike conventional search, a chatbot can maintain the thread of an inquiry, allowing us to explore a subject without starting again with every question. Research into conversational query understanding examines how systems interpret follow-up questions using the context of earlier exchanges.
The second development concerns where those conversations take place. Imagine having to move to another room every time a conversation developed a new direction. Similarly, switching between information systems can interrupt a researcher’s train of thought. Researchers can increasingly question documents, compare records, and analyze datasets within their existing work environment.
Harvard Kennedy School’s Modalities Beyond Chatbots explores how AI capabilities are becoming embedded in the applications people already use. For libraries, archives, and knowledge organizations, imagine being able to explore decades of reports, correspondence, and institutional records through an ongoing conversation.
The attraction is obvious: an inquiry can develop naturally, with relevant information close at hand. But an engaging conversation does not necessarily produce sound research. Someone still needs to assess the evidence, recognize what may be missing, and decide which connections are meaningful.
When Market Forces Meet Chatbots: A Legal Example
Legal technology offers a useful example of how chatbots are being used commercially. Law is heavily dependent on authoritative information, extensive document collections, and contextual interpretation. Research is time-consuming and expensive, which creates strong incentives to improve how information is found and used.
Recent market developments illustrate the move toward systems that connect natural-language interaction with legal research collections, document management, and repeatable workflows. Users can increasingly describe what they need, inspect retrieved material, compare documents, develop drafts, and initiate connected tasks within their working environment.
Competition is accelerating development, with specialist providers and major AI companies investing in increasingly sophisticated research capabilities. However, new capabilities and reported productivity gains do not necessarily establish better research outcomes. And as with any research tool, integration alone does not guarantee that people will want to use it.
When Can We Trust a Chatbot?
Anyone who has introduced a new knowledge management system knows that initial enthusiasm doesn’t guarantee lasting use. My LEXICON framework for end-user engagement emphasizes listening to users, understanding their working habits, and making systems genuinely useful. The same principles apply to chatbots, however appealing the conversational experience may be.
One legal technology company discovered this when its early self-service system struggled to attract users. People preferred asking their legal colleagues questions through familiar channels such as email, Slack, or Teams. The company subsequently developed AI agents that could work within those channels. The lesson is familiar: people are unlikely to change their habits simply because a new tool is available.
But even when people are happy to use a chatbot, does that mean it’s making their work easier? A 2025 study by Wang, Lu, and Pan found that conversational coherence was associated with less ambiguity, but not significantly less cognitive effort. A chatbot may be easy to talk to while still leaving users with considerable work to do in understanding the information and deciding what comes next.
Part of providing a trustworthy service is knowing when automated assistance is no longer enough. Library reference services have long combined automated help with access to a librarian. As conversational AI becomes more sophisticated, giving users a clear route to human expertise remains an important part of good service design.
Beyond the Answer
There is an important distinction between being trusted, being trustworthy, and being consulted. People may trust a chatbot because it is accessible, responsive, and confident. Yet trustworthiness depends on reliable evidence, an understanding of context, and knowing when an answer needs further investigation.
Information professionals have considerable experience helping people ask better questions, identify appropriate sources, and make sense of what they find. The challenge is ensuring that this expertise remains influential when users increasingly begin their research through conversational AI.
As chatbots become a more familiar route into knowledge collections, we need to consider what happens to the information people encounter along the way. Which sources are brought forward, which connections are made, and what might be overlooked? These questions lead us to the next article in this series, What Relevance Leaves Out.
0 Comments