Learn Generative AI and Large Language Models

Upskill to The Next Frontier of AI

Tutorial Overview

Fine Tuning Embedding Models

This workshop explores the importance of fine-tuning Language and Embedding Models (LLMs). It highlights how embedding models are used to map natural language to vectors, crucial for pipelines with multiple models to adapt to specific data nuances. An example demonstrates fine-tuning an embedding model for legal text. The notebook discusses existing solutions and hardware considerations, emphasizing GPU usage for large data.

The practical part of the notebook shows the fine-tuning process of the "distilroberta-base" model from the SentenceTransformer library. It utilizes the QQP_triplets dataset from Quora for training, designed around semantic meaning. The notebook prepares the data, sets up a DataLoader, and employs Triplet Loss to encourage the model to map similar data points closely while distancing dissimilar ones. It concludes by mentioning the training duration and resources needed for further improvements.

Tutorial Topics:

  • Fine Tuning 
  • Embedding Models
  • Map Natural language to Vectors
  • Model Training Duration

This course consists of an on-demand recording, course notebook, and course exercises

Before accessing the course code notebooks it is advisable to review the course prerequisites here.

Meet your instructor

Senior Machine Learning Engineer / Data Science Consultant

Mary Grace Moesta

Mary Grace Moesta is a senior data science consultant at Databricks. She's been working in the big data and data science space for several years with opportunities to collaborate across several verticals, with the majority of her work focused in the Retail and CPG space. Prior to Databricks, Mary Grace was able to contribute to several machine learning applications, namely - personalization use cases, forecasting, recommendation engines, and customer experience measures.

Course Curriculum

  • 1

    Welcome to the Tutorial!

    • Welcome to the tutorial!

    • What You'll Learn in This Tutorial

    • How to use this tutorial

    • Tutorial Prerequisites

  • 2

    Fine Tuning Embedding Models

    • Fine Tuning Part I: Embedding Models

    • Lesson Notebook: Fine Tuning Embedding Models

    • Fine Tuning Quiz


A Hands-on Tutorial

This hands-on tutorial goes beyond the basics, offering you an interactive Coding Notebook crucial to your educational journey. It immerses you in an engaging process of writing, generating, and executing code, enabling a comprehensive exploration of the tutorial's core concepts through practical coding exercises. By applying these concepts in real time, you'll witness the immediate impact of your coding choices. This hands-on approach is not just about learning to code; it's about coding to learn, solidifying your understanding as you seamlessly generate and execute code.

Enroll now!

Accelerate your journey to enerative AI by enrolling in our program today!