Description

Talk

This session introduces the AI Gateway pattern—a critical architectural component that serves as the central control plane for enterprise AI ecosystems. We'll explore how AI gateways solve real-world challenges through unified API abstraction, intelligent failover mechanisms, semantic caching, centralized guardrails, and granular cost controls. You'll learn practical architectural patterns for building high-availability gateways that handle thousands of concurrent requests while maintaining sub-millisecond decision-making through in-memory operations. We'll dive deep into the technical architecture, covering separation of control and data planes, asynchronous logging patterns, and horizontal scaling strategies.

Instructors Bio

Nikunj Bajaj

Co-founder and CEO of TrueFoundry

Nikunj is the co-founder and CEO of TrueFoundry, a platform helping enterprises build, deploy and ship LLM applications in a fast, scalable, cost efficient way with right governance controls within their own cloud. Prior to this role, he served as a Tech Lead for Conversational AI at Meta, where he spearheaded the development of proactive virtual assistants. His team also put Meta's first deep learning model on-device. Nikunj also led the Machine Learning team at Reflektion, where he built an AI platform to enhance search and recommendations for over 600 million users across numerous eCommerce websites. He has done his bachelors in Electrical Engineering from IIT Kharagpur and masters in Computer Science from UC Berkeley.

Dozens of Free Courses with Premium

Unlock Premium Features with a Subscription

  • Live Tarining:

    Full access to all live workshops and training sessions.

  • 20+ Expert-Led Workshops:

    Dive deep into AI Agents, RAG, and the latest LLMs

  • ODSC Conference Discounts:

    Receive extra discounts to attend ODSC conferences.