Example: PDF Chatbot📚

Description:

This example showcases how to build a PDF chatbot with local LLM and Embedding models

Used Technology:

@ Xinference as a LLM model hosting service

@ LlamaIndex for orchestrating the entire RAG pipeline

@ Streamlit for interactive UI

Detailed Explanation on the Demo Functionality :

• Crafted a Dockerfile to simplify the process and ensure easy reproducibility.

• Set up models with Xinference and expose two ports for accessing them.

• Leverage Streamlit for seamless file uploads and interactive communication with the chat engine.

• 5x faster doc embedding than OpenAI’s API.

• Leveraging the power of GGML to offload models to the GPU, ensuring swift acceleration. Less long waits for returns.

Source Code :

• PDF Chatbot