Fine Tuning Embedding Models
This workshop explores the importance of fine-tuning Language and Embedding Models (LLMs)
Tutorial Overview
Fine Tuning Embedding Models
This workshop explores the importance of fine-tuning Language and Embedding Models (LLMs). It highlights how embedding models are used to map natural language to vectors, crucial for pipelines with multiple models to adapt to specific data nuances. An example demonstrates fine-tuning an embedding model for legal text. The notebook discusses existing solutions and hardware considerations, emphasizing GPU usage for large data.
The practical part of the notebook shows the fine-tuning process of the "distilroberta-base" model from the SentenceTransformer library. It utilizes the QQP_triplets dataset from Quora for training, designed around semantic meaning. The notebook prepares the data, sets up a DataLoader, and employs Triplet Loss to encourage the model to map similar data points closely while distancing dissimilar ones. It concludes by mentioning the training duration and resources needed for further improvements.
Tutorial Topics:
- Fine Tuning
- Embedding Models
- Map Natural language to Vectors
- Model Training Duration
This course consists of an on-demand recording, course notebook, and course exercises
Before accessing the course code notebooks it is advisable to review the course prerequisites here.
Senior Machine Learning Engineer / Data Science Consultant
Mary Grace Moesta
Mary Grace Moesta is a senior data science consultant at Databricks. She's been working in the big data and data science space for several years with opportunities to collaborate across several verticals, with the majority of her work focused in the Retail and CPG space. Prior to Databricks, Mary Grace was able to contribute to several machine learning applications, namely - personalization use cases, forecasting, recommendation engines, and customer experience measures.
-
1
Welcome to the Tutorial!
-
Welcome to the tutorial!
-
What You'll Learn in This Tutorial
-
How to use this tutorial
-
Tutorial Prerequisites
-
-
2
Fine Tuning Embedding Models
-
Fine Tuning Part I: Embedding Models
-
Lesson Notebook: Fine Tuning Embedding Models
-
Fine Tuning Quiz
-
CODE TO LEARN
A Hands-on Tutorial
This hands-on tutorial goes beyond the basics, offering you an interactive Coding Notebook crucial to your educational journey. It immerses you in an engaging process of writing, generating, and executing code, enabling a comprehensive exploration of the tutorial's core concepts through practical coding exercises. By applying these concepts in real time, you'll witness the immediate impact of your coding choices. This hands-on approach is not just about learning to code; it's about coding to learn, solidifying your understanding as you seamlessly generate and execute code.
