Retrieval Augmented Generation - RAG

RAG is a simple three-step process: indexing, retrieval, and generation.

LlamaIndex or LangChain?

In straightforward terms, LLamaIndex plays a crucial role in retrieving context pieces and interacting with the LLM. It shares similarities with LangChain in the overall process, albeit with distinctive terminology: LLamaIndex refers to “chunks” as “nodes”. ¹

But, what sets LLamaIndex apart from LangChain, and how can you make an informed choice between the two?

• In a nutshell: ¹
  • LLamaIndex: offers a plethora of options for processing/chunking various document types and it provides a rich array of retrieval possibilities.
  • LangChain: demonstrates more flexibility, providing extensive options for interaction with the LLM

DSPy vs LangChain & LlamaIndex ²

LangChain and LlamaIndex specialise in high-level application development, offering ready-to-use application modules that seamlessly integrate with your data or configuration.

If you prefer using a standard prompt for tasks like question answering over PDFs or converting text to SQL, these libraries provide a comprehensive ecosystem to explore.

In essence, Declarative Self-Improving Language Programs (DSPy) is designed for scenarios where you require a lightweight, self-optimising programming model rather than relying on pre-defined prompts and integrations.

Enhancing data granularity

Metadata is useful because it brings an additional layer of structured search on top vector search. ³

Hyde or Query2doc

Both Hyde and Query2doc are similar query rewriting optimisations. Given that search queries are often short, ambiguous, or lack necessary background information, LLMs can provide relevant information to guide retrieval systems, as they memorize an enormous amount of knowledge and language patterns by pre-training on trillions of tokens. ³

The essence of HyDE (Hypothetical Document Embeddings) is to use LLM to generate hypothetical documents for user queries. These documents are generated based on the knowledge of LLM itself and may contain errors or inaccuracies. However, they are associated with the documents in the knowledge base of RAG. Then, by using these hypothetical documents to retrieve real documents with similar vectors, the accuracy of retrieval is improved. ⁴

^{difference between Standard and Hyde approach}

other references

• RAG usage on an internal knowledge base
• Choosing between LlamaIndex or LangChain

---

1. Offline RAG with LlamaIndex and tiny/small LLMs ↩ ↩²

2. An Introduction To DSPy ↩

3. Advance RAG- Improve RAG performance ↩ ↩²

4. Advanced RAG Retrieval Strategy: Query Rewriting ↩