A Fun Weekend Project: Q&A Chatbot Trained on My Own Content

Any book can now be transformed into a chatbot that can answer questions about its content. It doesn’t to have to be a book either. It can also be your own content. For example, you can use your own notes or favorite articles and ask any question about what’s mentioned in them — just as you would ask ChatGPT.

You can do this by fine-tuning OpenAI’s language models (GPT) on your own data. All you will need is the OpenAI API and two open-source libraries (LlamaIndex and Langchain). You can run this in a Google Colab (Jupyter) notebook for free. If this sounds interesting, here is how:


1. Open AI. Sign up for Open AI and get an API key.

2. Upload your content to GitHub.

3. Use a Colab notebook.

  • To simplify things, we’ll use Google Colab. It runs in the cloud, so you won’t need to install anything on your laptop.
  • To get started quickly, re-use a public Colab that has all the necessary code. This way you would only need to make a few changes. For example, you can copy this Colab (credit goes to Dan Shipper) to your Google Drive. For context, see Dan’s article in Lenny’s newsletter and this video walkthrough by Irina Nik.
  • Replace the GitHub link with your own.
  • Change runtime type to GPU (Menu: Runtime -> Change runtime type).
  • Run all the Colab cells.
  • Enter your OpenAI API code when prompted.
  • Ask any question when prompted.

And that’s it!

You can find a copy of the complete code below:

! git clone https://github.com/[insert your repo].git
!pip install llama-index
!pip install langchain
from llama_index import SimpleDirectoryReader, GPTListIndex, readers, GPTSimpleVectorIndex, LLMPredictor, PromptHelper
from langchain import OpenAI
import sys
import os
from IPython.display import Markdown, display

def construct_index(directory_path):
    # set maximum input size
    max_input_size = 4096
    # set number of output tokens
    num_outputs = 2000
    # set maximum chunk overlap
    max_chunk_overlap = 20
    # set chunk size limit
    chunk_size_limit = 600 

    # define LLM
    llm_predictor = LLMPredictor(llm=OpenAI(temperature=0.5, model_name="text-davinci-003", max_tokens=num_outputs))
    prompt_helper = PromptHelper(max_input_size, num_outputs, max_chunk_overlap, chunk_size_limit=chunk_size_limit)
    documents = SimpleDirectoryReader(directory_path).load_data()
    index = GPTSimpleVectorIndex(
        documents, llm_predictor=llm_predictor, prompt_helper=prompt_helper


    return index

def ask_ai():
    index = GPTSimpleVectorIndex.load_from_disk('index.json')
    while True: 
        query = input("What do you want to ask? ")
        response = index.query(query, response_mode="compact")
        display(Markdown(f"Response: <b>{response.response}</b>"))
os.environ["OPENAI_API_KEY"] = input("Paste your OpenAI key here and hit enter:")

Leave a Reply