Case Study
LLM/RAG Assistant with Tool Calling
Based on production systems built at Infidea.
Summary
Built a production RAG system for an aesthetic clinic's patient intake workflow at Infidea. Combined LlamaIndex document ingestion, Qdrant vector search, LangGraph orchestration with tool calling for appointment booking and structured data collection, and LiveKit/GPT-Realtime for voice-based patient interactions. The system had to handle real-time client-facing conversations with low latency and reliable structured outputs.
What This Page Covers
- How the RAG pipeline handles document ingestion and retrieval across multiple source types
- Why tool calling was needed and how it was integrated into the LangGraph workflow
- Architecture decisions for reliability in a healthcare-adjacent, client-facing context
Approach
The system was built in three layers: ingestion, retrieval, and orchestration. For ingestion, I used LlamaIndex to chunk and embed client documents and PDFs, and Playwright to scrape client websites with JavaScript-rendered content. Qdrant handled vector storage with metadata filtering for document source and type. The orchestration layer used LangGraph to manage the conversation flow, with structured tool calling for appointment booking (Calendly/Google Calendar integration) and information intake forms.
Voice-based interactions used LiveKit for real-time WebRTC audio and GPT-Realtime for streaming speech-to-speech responses. FastAPI served as the backend, connecting the RAG pipeline, tool calls, and voice infrastructure.
Key Decisions
- Qdrant over FAISS — production-ready filtering, easy horizontal scaling, and better integration with LlamaIndex
- Playwright for scraping — many client websites use JS-rendered content that plain HTTP requests miss
- LangGraph over raw LangChain — explicit state management and branching logic for tool calling workflows
- Structured output schemas — tool definitions with strict schemas to ensure reliable data collection from the LLM
Tech Stack
Python, FastAPI, Qdrant, LlamaIndex, LangGraph, LangChain, OpenAI API, Playwright, LiveKit, GPT-Realtime, tool calling.
Outcome
Production system running patient intake and coordination workflows for aesthetic clinic operations. Handles voice interactions, document-grounded Q&A, and structured appointment booking in a single pipeline.
Need something like this?
If you are building a RAG system, document assistant, or tool-calling LLM workflow, contact me to discuss the architecture.