You are here

Home

Using AI to Determine the Political Point of View of Books

Posted by Jim Craner on March 14, 2024

Author's Note: This is one of those areas where information has both positive and negative applications.  As a former elected library board member for almost a decade, I believe libraries should represent a wide spectrum of voices, perspectives, and experiences.  In addition to being against any type of book ban, I always encouraged our library staff to select books from across the political spectrum, even from the end of the spectrum opposite mine. How do we ensure that our selection practices are fair and diverse if we don't measure the results?

On the other hand, there are a lot of fascists and busybodies out there trying to ban books. Most of them hail from one end of the political spectrum and have a lot of animosity for books "representing" the other end of that spectrum. I don't want to provide any "cover" or tools for these folks.

With that said, let's talk about using AI to determine the political point of view of books in our collections!

Earlier this week, I read this great practical blog post from Elissa Malespina on using AI, specifically ChatGPT, to create book synopses for shelf talkers (which is a little tag or card on the shelf describing a featured book).

At this point last year, I would have hesitated about using AI in this manner for two main reasons:

Information Accuracy ("Hallucinations")

Anyone who has taken even one of my AI courses has heard me say this repeatedly: AI models are NOT accurate factual databases!  This is especially true when we are talking about obscure information, or information that wasn't present in the AI's training materials.  ChatGPT will almost always correctly tell you that Paris is the capital of France, for instance.  However, as one school district in Iowa found out last year, ChatGPT is not an authoritative source of the complete content of every book every written.  And when ChatGPT doesn't "know" something, it tends to just confidently make up nonsense that "sounds" correct.

Knowledge Cut-off

When ChatGPT was first released in early 2023, it was only "trained" on information through September 2021 (with some exceptions). This meant it was unaware of events after that cutoff date.  For our readers advisory purposes, an 18-month gap in "current events" means that the AI model wouldn't be aware of more recent releases. 

Since then, however, AI models have been brought "up to date" with more recent information.  The latest GPT-4 versions, as of the time of this writing in March 2024, are trained on data through December 2023 (although ChatGPT itself, as of today, uses an earlier model with an April 2023 cutoff).  In addition, ChatGPT Plus uses integrated web searching to find more recent data when necessary (although this has some potential gotchas, as well).

Then a student from one of our recent AI training workshops for library staff reached out to me with a related request: given a list of book titles, could an AI model accurately provide the author's or item's point of view on the political spectrum.  Great question, let's find out!

Test #1, Feasibility

First I wanted to test basic feasibility, to make this as easy as possible for ChatGPT.  So I picked out some book titles where virtually everyone could agree on the book's political point of view.  I did this very lazily: I copied and pasted Al Franken's bibliography and Rush Limbaugh's bibliography from Wikipedia.  Then I added a few non-political kids books and sci-fi books that I personally have read recently.

Test Titles
  • I'm Good Enough, I'm Smart Enough, and Doggone It, People Like Me!: Daily Affirmations By Stuart Smalley
  • Rush Limbaugh Is a Big Fat Idiot and Other Observations
  • Why Not Me?: The Inside Story of the Making and Unmaking of the Franken Presidency
  • Oh, the Things I Know!: A Guide to Success, or Failing That, Happiness
  • Lies and the Lying Liars Who Tell Them: A Fair and Balanced Look at the Right
  • The Truth (With Jokes)
  • Pete the Cat
  • From the Mixed Up Files of Mrs Basil E Frankweiler
  • The 3-Body Problem
  • Cryptonomicon
  • The Grapes of Wrath
  • Green Eggs and Ham
  • The Way Things Ought to Be
  • See, I Told You So
  • Rush Revere and the Brave Pilgrims
  • Rush Revere and the First Patriots
Prompt

Describe the point of view, on the political spectrum, of the following books. Return a list of just the titles and then below each title, a single brief phrase describing the political point of view. Bold the titles so it's easier to read the results.

Results
I'm Good Enough, I'm Smart Enough, and Doggone It, People Like Me!: Daily Affirmations By Stuart Smalley
Satirical self-help
Rush Limbaugh Is a Big Fat Idiot and Other Observations
Liberal satire
Why Not Me?: The Inside Story of the Making and Unmaking of the Franken Presidency
Political satire from a liberal perspective
Oh, the Things I Know!: A Guide to Success, or Failing That, Happiness
Humorous self-help, politically neutral
Lies and the Lying Liars Who Tell Them: A Fair and Balanced Look at the Right
Liberal critique of conservative media
The Truth (With Jokes)
Liberal political satire
Pete the Cat
Children's literature, politically neutral
From the Mixed Up Files of Mrs. Basil E. Frankweiler
Children's adventure, politically neutral
The 3-Body Problem
Science fiction, politically neutral with themes of political history in China
Cryptonomicon
Science fiction with historical and technological themes, politically neutral
The Grapes of Wrath
Critique of capitalism, leans left
Green Eggs and Ham
Children's literature, politically neutral
The Way Things Ought to Be
Conservative viewpoint
See, I Told You So
Conservative viewpoint
Rush Revere and the Brave Pilgrims
Conservative, historical fiction for children
Rush Revere and the First Patriots
Conservative, historical fiction for children

Analysis

Not bad! Again, this was just a feasibility test but it nailed it. A couple of interesting things:

  • It called Grapes of Wrath "leans left" which I tried to duplicate. On the second run, it gave me "Critique of American capitalism, pro-worker" which was interesting.
  • I have no clue how 3-Body Problem was received in Chinese political circles but I will try to find out when I re-read it.

Next Steps

So that was an easy test: relatively well-known books with clear political bias (or lack thereof) that have been discussed for years online, giving the AI model plenty of opportunities to "absorb" information about them.  But the list of books provided by our library director friend for review included:

  • a lot more titles
  • all published in 2021 and 2022, so with presumably less online discussion and reviews from which the AI can "learn" 
  • titles that weren't necessarily "political" in topic or nature

So over the next few days, we'll be working on some methods to test whether this sort of automated ideological assessment is possible with AI tools, and then how well it works.  Stay tuned!