You are here

Home

Using "Retrieval" To Enhance ChatGPT's Capabilities

Posted by Jim Craner on April 30, 2023

My colleagues can confirm that I'm the first person to rave about how useful ChatGPT has been for my work. I've been using it since it became available for tons of everyday tasks: writing, coding, strategizing, brainstorming, and more. But a lot of the recent hype around AI tools, especially ordinary use of the basic ChatGPT system from OpenAI, really needs to be taken with a grain of salt!

Anyone who has attended our Public Library & AI Demonstration, Discussions, and Discovery events knows that ChatGPT and other similar "Large Language Model"-based ("LLM") AI tools are great at certain language based tasks - but absolutely terrible at other tasks. The basic strengths of LLM tools fall into these categories:

  • "understand" text
  • "classify" text, grouping, based on sentiment or other characteristics
  • "summarize" text, based on "understanding"
  • "extraction" of information from text
  • "transformation" of text (e.g., translation)
  • "generate" text based on a prompt ("completion")
  • "conversation" based on a prompt

-- from the PLAID3 website

These are incredibly useful strengths, and we're always thinking of new ways to combine them. But ChatGPT definitely has some weaknesses as well:

Knowledge Cut-off

ChatGPT, as of the time of this writing, has a knowledge cut-off of September 2021. That means that the "trainers" of AI only used information available in September 2021 to "teach" ChatGPT, and it will have limited, if any, knowledge about events that happened after that point.

Chat Transcript. User: What were the books on the NYT Bestseller list Top 5 in December 2022? AI:I'm sorry, but my training data only goes up until September 2021, so I cannot provide you with the specific information about the New York Times Bestseller list for December 2022. However, I recommend checking the New York Times website or other reliable sources for the most recent and accurate information on best-selling books.

Plugin Woes

ChatGPT has added plugins, which are "add-on tools" that can be installed that let the Chat application interact with other websites or services. For instance, you can enable the Kayak plugin which allows ChatGPT to search the Kayak.com flight reservation search engine on your behalf. Plugins have a ton of potential, but right now, there are only a few "approved" plugins and they're only available to a certain subset of ChatGPT users.

Chat window showing the "Using KAYAK" plugin button. Then an error message: UnrecognizedKwargsError: cabinClass I apologize for the inconvenience. It seems that I am unable to search for first-class flights using the flight insights feature. However, I can still help you find the cheapest flights

Hallucinations

ChatGPT knows a LOT -- after all, its training regimen included reading the majority of the public Internet. But ChatGPT also "hallucinates," which is the AI term for "just making up complete nonsense, but stating it confidently."  Anyone who's used ChatGPT for a while has encountered one of these hallucinations -- and the confident nature with which ChatGPT makes these made-up statements.  Hallucinations keep ChatGPT from being a reliably useful search engine, discovery layer or readers advisory resource, or many other tasks for which accuracy of information is paramount.

Let's look at an example of an AI making a truthful statement and a hallucination to illustrate this.

When testing AI-based tools, what we refer to as "intelligent tools," we frequently use the Project Gutenberg library of public domain books as test data. Let's look at the classic "Anne of Green Gables," by Lucy M. Montgomery for our example. Anyone who's read the book might remember a famous anecdote about the raspberry cordial: teenage Anne mistakenly serves alcoholic currant wine to her friend and herself instead of the raspberry cordial soft drink that she intended. Her friend gets tipsy and both girls get in trouble. This is a popular scene in a popular book, so naturally ChatGPT has "read about it" and can accurately share what happened.

chat window. User asks: What's the deal with Anne of Green Gables and the raspberry cordial? AI responds: Anne of Green Gables is a classic novel written by L.M. Montgomery, first published in 1908. It tells the story of Anne Shirley, a young orphan girl with a vivid imagination, who is mistakenly sent to live with siblings Marilla and Matthew Cuthbert on their farm, Green Gables, on Prince Edward Island in Canada. The raspberry cordial incident is a significant and humorous episode in the story. Anne invites her new friend, Diana Barry, over to Green Gables for an afternoon tea. Anne wants to be the perfect hostess and decides to serve what she believes to be raspberry cordial, a non-alcoholic drink made from raspberries, sugar, and water. However, Anne mistakenly grabs a bottle of currant wine instead of raspberry cordial, and both girls drink several glasses of it. Diana becomes intoxicated, and her mother is furious when she finds out. As a result, she forbids Diana from associating with Anne, believing her to be a bad influence. Anne is devastated by the loss of her friendship with Diana and feels terrible about the mistake. The incident eventually leads Anne to reflect on her impulsiveness and the importance of being

So far, so good, right? What if we ask ChatGPT about something less well-known?  Like the slippers Anne got for Christmas that year?

chat conversation.user asks: describe the slippers Anne got for Christmas. AI responds: In "Anne of Green Gables," Anne receives a pair of lovely soft, furry slippers for Christmas from Matthew Cuthbert, who has grown fond of her. These slippers are described as being made of red cloth, with fur trimming around the top, and a ribbon bow on the front. The slippers are a symbol of the love and acceptance Anne starts to experience from her new family at Green Gables. The gift is especially meaningful because it is one of the first times Anne receives a Christmas present, and it is a thoughtful, personal gift that shows Matthew's understanding of Anne's love for beautiful things. The slippers also reflect the growing bond between Anne and Matthew, which becomes one of the most heartwarming aspects of the novel.

Uh oh! This is a complete fabrication, folks. Or, in AI terms, a "hallucination." Readers of chapter 25 will remember that, in the actual book:

  • the slippers come from Diana's Aunt Josephine, not from Matthew;
  • the slippers are made from kid (goat skin), not furry red cloth;
  • Matthew gave Anne the dress with the puffy sleeves.

“Merry Christmas, Diana! And oh, it’s a wonderful Christmas. I’ve something splendid to show you. Matthew has given me the loveliest dress, with such sleeves. I couldn’t even imagine any nicer.” “I’ve got something more for you,” said Diana breathlessly. “Here—this box. Aunt Josephine sent us out a big box with ever so many things in it—and this is for you. I’d have brought it over last night, but it didn’t come until after dark, and I never feel very comfortable coming through the Haunted Wood in the dark now.” Anne opened the box and peeped in. First a card with “For the Anne-girl and Merry Christmas,” written on it; and then, a pair of the daintiest little kid slippers, with beaded toes and satin bows and glistening buckles.

Intelligent Tools to the Rescue!

So, how do we get around these problems? Like a lot of technology teams, we are hard at work building AI-enabled intelligent tools using these new AI services. We've done a lot of hard work in this area, trying to envision how library staff - and patrons - could benefit from some of these tools. And we want to make sure these tools are:

  • able to access up-to-date information from after the general ChatGPT training cutoff date;
  • reliable and easy to use, even for staff without ChatGPT access
  • trustworthy, based on legitimate information

Here's a screenshot from an early version of our Document Analysis tool, displaying the correct answer to the question.

screenshot of custom AI application, titled "Collection: Anne of Green Gables, 278 documents" - Form reads "Ask a question" with question "Describe the slippers Anne got for Christmas." Below the form is an answer, circled in red, reading Anne got a pair of dainty little kid slippers with beaded toes, satin bows, and glistening buckles for Christmas from Aunt Josephine as mentioned in the relevant document

So what's the secret?  Apps like this use a strategy called "retrieval" to provide relevant information to ChatGPT that it will need to answer the question.  Here's a simple explanation of how it works.

First, we upload the document into a special kind of database. AI systems can't handle "reading" (or processing) an entire book at one time so documents are split into sections, like pages.

graphic showing an antique copy of Anne of Green Gables with pages coming out of it

Next, each piece of the document is analyzed and transformed into a complex series of numbers using advanced math. Modern AI systems like ChatGPT are "large language models" ("LLMs") but underneath? It's all numbers :-)

image showing chapter 1, page 1, of the book, subtitled "Mrs. Rachel Lynde is Surprised." On the right is a long column of numbers running parallel to the book

Now when you ask the system a question, your question is used to find relevant pages in the database that might answer the question. These pages, and your original question, are then given to ChatGPT. We also usually include a specific prompt, such as "Given the context of these pages, answer the following question." 

image showing a page from ANne of Green Gables with the word "slippers" highlighted twice. On the right are two chat bubble symbols. The top bubble reads "Using this text, answer this question:" and the bottom bubble reads "Describe the slippers Anne got for Christmas"

ChatGPT can now correctly answer the question without hallucinating!

screenshot of custom AI application, titled "Collection: Anne of Green Gables, 278 documents" - Form reads "Ask a question" with question "Describe the slippers Anne got for Christmas." Below the form is an answer, circled in red, reading Anne got a pair of dainty little kid slippers with beaded toes, satin bows, and glistening buckles for Christmas from Aunt Josephine as mentioned in the relevant document

 

Are you using AI in your library or information services operations? Let us know! Interested in learning more about AI and intelligent tools? Check out our AI workshop series, beginning in May 2023!