API World 2023 + AI DevWorld 2023

Daniel Huynh, Mithril Security, CEO

Large Language Models have become the new hot topic, as models such as Falcon have proven their efficiency in answering a wide range of questions, from code analysis to medical answering, through email summarization. However, sending sensitive data to AI vendors creates privacy risks as control of data usage becomes complicated.

In this talk, we will explore how confidential computing can enable users to benefit from AI model predictions without ever having to disclose their data in an unencrypted form, ensuring compliance.

We will demonstrate how you can use confidential containers to serve AI models with privacy guarantees for user data, and how these containers can be employed to wrap Docker images with isolation layers, deploying them inside confidential containers. This effectively replicates the security of an on-premises, air-gapped environment while maintaining the scalability of SaaS.

Benefits to the ecosystem:
We assist in imparting knowledge on implementing LLMs with on-premises-level control and end-to-end security through Confidential Computing. No one can access the data owners' information, which fosters trust via a zero-trust model and ensures effortless compliance.

We hope this will help foster the adoption of AI in sensitive industries like healthcare or finance, where compliance issues must be addressed.

Key takeaways from this presentation include:
1. Understanding effective strategies to mitigate data exposure during analysis.
2. Gaining a comprehensive understanding of Confidential Computing and its practical applications.
3. Learning the process of deploying Large Language Models (LLMs) within confidential containers to enhance privacy protection.
4. Applying the presented strategies and techniques to safeguard data and maintain privacy in future computing endeavors.