Course Abstract

Training duration: 90 min (Hands-on)

In this 90-minute course, Ankur Patel will provide an in-depth take on modern NLP, retracing how NLP advanced over the last decade and experienced its breakout moment in 2018. Since then, NLP has soared in popularity among companies and become a mainstream topic of interest. After we cover the theory, we will discuss modern NLP tasks such as sequence classification, question answering, language modeling, text generation, named entity recognition, summarization, and translation. These tasks underpin many of the powerful NLP applications on the market today. To conclude, we will develop our own named entity recognition and text classification models using spaCy, including annotating our data using an annotation platform called Prodigy. We will perform transfer learning and fine-tuning and compare the fine-tuned model’s performance against an out-of-the-box named entity recognition model.

DIFFICULTY LEVEL: INTERMEDIATE

Instructor Bio:

Ankur Patel

Co-founder and Head of Data | Glean

Ankur Patel

Ankur Patel is the co-founder and Head of Data at Glean. Glean uses NLP to extract data from invoices and generate vendor spend intelligence for clients. Ankur is an applied machine learning specialist in both unsupervised learning and natural language processing, and he is the author of Hands-on Unsupervised Learning Using Python: How to Build Applied Machine Learning Solutions from Unlabeled Data and Applied Natural Language Processing in the Enterprise: Teaching Machines to Read, Write, and Understand. Previously, Ankur led teams at 7Park Data, ThetaRay, and R-Squared Macro and began his career at Bridgewater Associates and J.P. Morgan. He is a graduate of Princeton University and currently resides in New York City.

Course Outline

Module 1: Modern NLP in theory

  • The path to NLP’s watershed “ImageNet” moment in 2018
  • Word embeddings: one-hot encoding, word2vec, GloVe, fastText, and context-aware pretrained word embeddings
  • Sequential models: vanilla recurrent neural networks (RNNs), long short-term memory (LSTMs), and gated recurrent units (GRUs)
  • Attention mechanisms and Transformers
  • ULMFiT, ELMo, BERT, BERTology, GPT-1, GPT-2, and GPT-3


Module 2: Modern NLP in action

  • Refresher on pre-trained language models, transfer learning, and fine-tuning
  • Introduction to common NLP tasks via Hugging Face: sequence classification, question answering, language modeling, text generation, named entity recognition, summarization, and translation


Module 3: Modern NLP applications

  • Explore dataset: AG news dataset
  • Application #1: Named Entity Recognition (NER)
  • Perform inference using out-of-the-box spaCy NER model
  • Annotate data using Prodigy
  • Develop custom named entity recognition model using spaCy
  • Compare custom NER model against the out-of-the-box spaCy NER model
  • Application #2: Text Classification
  • Annotate data using Prodigy
  • Develop text classification model using spaCy


Conclusion

Background knowledge

  • Python coding experience

  • Familiarity with pandas, numpy, and scikit-learn

  • Understanding of basic machine learning concepts, including supervised learning

  • Experience with deep learning and frameworks such as TensorFlow or PyTorch is a plus

Applicable Use Cases

  • Semantic Search: search the entire web or a repository of documents and surface relevant search results (e.g., Google Search)

  • Natural Language Generation: auto-complete sentences as you write emails or draft documents (e.g., Gmail)

  • Machine Translation: convert text and audio from one language to another (e.g., Google Translate and Apple Translate)

  • Speech Recognition and Question Answering: give voice commands and control your home devices (e.g., Amazon Alexa, Apple Siri, Google Assistant, and Microsoft Cortana)

  • Customer Service Chatbots: ask account-related questions and get mostly reasonable answers (e.g., Intercom)