Chat with Your Book
STOP! Please read this version of the article on Medium instead! That is the article I wrote, the one below has been rephrased by ChatGPT.'
As an author, I naturally encourage people to read books, including my own. However, the sheer volume of books available can be overwhelming. Have you ever faced a homework assignment for a book you despise? Or perhaps you're contemplating purchasing a specific book but are concerned it might lack the depth you seek. If you're an author yourself, looking for comparative works for your latest novel (known as "comps"), the prospect of reading through numerous novels can be time-consuming and costly.
Instead of traditional reading, consider engaging in a conversation with a book. This transition is made relatively simple through the capabilities of Generative AI and Large Language Models (LLMs).
An LLM like Chat-GPT or Palm 2 possesses the ability to comprehend natural language, essentially enabling it to "read" a book, understand your inquiries, and respond in plain language.
By harnessing the remarkable features of LLMs and utilizing a vector database, I've created a solution that allows you to pose questions about the plot and characters of my unpublished science fiction novel, "The Council of Light." If you decide to test this solution, be aware that it may reveal plot details, leading to spoilers.
Solution Overview
For more detailed information, read the full article on this solution on Medium, or you can access my GitHub repository and Colab. Access to the PDF of my book can be requested. My solution is based on the Retrieval Augmented Generation (RAG) framework, eliminating the need for fine-tuning or any adjustments to make the LLM aware of my book.
Before harnessing the LLM's capabilities, it's essential to provide domain-specific information, ensuring the LLM can effectively address questions. "The Council of Light" is a 140,000-word book, approximately the length of JRR Tolkien's "The Return of the King." Scanning the entire book every time a question arises would be highly inefficient and confusing. Furthermore, the LLM must derive meaning from each sentence, and it operates on numbers rather than words.
This is where embeddings play a vital role. Generating embeddings involves creating numerical representations not only for individual words, but also for the relationships between words within the text. Fortunately, the text-bison LLM in use is already well-versed in the English language, thanks to its training on a vast corpus of embeddings.
Consider the sentence from "The Council of Light": "Ben read the report on his com-palm." While the LLM can identify 'Ben' as a name and understand that humans read reports, essays, or books, it encounters the word 'com-palm' for the first time. The solution is to provide the LLM with additional embeddings. This is achieved by dividing the book into sections or passages and using Google's text embeddings API to generate a set of embeddings for each section. The hope is that the API's algorithm can determine the most pertinent information in each section, its relative importance, and represent this information in embeddings.
The subsequent step involves depositing these book embeddings into a vector store, specifically Vertex Vector Search. This facilitates semantic searches over the embeddings. If a user's question pertains to 'com-palms,' the vector store will return relevant text passages mentioning 'com-palms.'
This approach works well for searching encyclopedia-style texts like textbooks, but can it handle the intricate details and themes found in novels? For instance, consider the question, "who is the villain in the novel?" Novels typically don't label the antagonist as "the villain." While the search algorithm is smart enough to identify similar words like "evil" or "bad," answering such questions often requires more context.
Once I had loaded my vector store with the book's information, I used a Retrieval QA chain to bring everything together. The prompt I used looked like this:
You are an intelligent assistant helping the users with their questions about the plot and characters of a novel.
Strictly use ONLY the following pieces of context to answer the question at the end. Think step-by-step and then answer.
Do not try to make up an answer:
- If the answer to the question cannot be determined from the context alone, say "I cannot determine the answer to that."
- If the context is empty, just say "I do not know the answer to that."
=============
{context provided from the vector store}
=============
Question: {question asked by user}
Remember, you are an intelligent assistant helping the users with their questions about the plot and characters of a novel.
Helpful Answer:"""
Let's see it in action (note: there are some minor spoilers).
[I will just keep the first three query:response pairs from the original blog]
Want to see more responses from the LLM? Read the full article on Medium!
