API World 2023 + AI DevWorld 2023

Zain Hasan, Weaviate, Senior Developer Advocate

LLMs have revolutionized how we access and consume information shifting the pendulum from a search engine market that was predominantly retrieval based (where we asked for source documents containing concepts relevant to our search query) to one now that is growingly memory based and performs generative search (where we ask LLMs to generate answers to questions based on their knowledge of and training on massive datasets). Both have their pros and cons.

This talk discusses a hybrid approach, retrieval augmented generation (RAG), that gives us the best of both worlds. RAG allows us to generate language from LLMs grounded in source context provided by relevant documents retrieved by a vector database at scale and in production, over 100 Millions of documents in real time.

This approach is akin to allowing your LLM to go to the library and read a book relevant to the prompt prior to answering your prompt. It allows retrieval to act as non-parametric programmable memory for your LLM which can be used to control, hone and direct the raw power that LLMs like GPT 4.0 with parametric memory alone possess.

This not only allows your LLM to learn from your custom and proprietary data but also means that it will outperform on very specific technical tasks or on information that is underrepresented in the training set. Augmenting the parametric memory of LLMs with non-parametric memory also addresses the all-important problem of reducing false information generation.

We will conclude the talk with a code demonstration of the power of retrieval augmented generation to solve very specific tasks using LLMs(GPT4.0 and open source versions) and vector databases at scale!