API World 2023 + AI DevWorld 2023

Philip Tannor, Deepchecks, Co-Founder & CEO

In this talk, we'll delve into how to handle the challenges and risks of integrating Large Language Models (LLMs) in production applications. Amidst the promise of LLMs, their stochastic nature, tendency for hallucinations, potential biases, and the issue of robust, affordable evaluation pose significant challenges. We'll explore methods for creating solid evaluation sets, efficient evaluation of product versions, and comprehensive strategies for testing to counteract inherent model weaknesses, including bias, hallucinations, prompt injection attacks, topic deviations, and harmful output. By addressing these testing challenges and establishing rigorous requirements, we can maximize the potential of LLMs while reducing associated risks.