Natural Language Processing: NLP With Transformers in Python

Natural Language Processing: NLP With Transformers in Python

English | MP4 | AVC 1920×1080 | AAC 44KHz 2ch | 11.5 Hours | 3.61 GB

Learn next-generation NLP with transformers for sentiment analysis, Q&A, similarity search, NER, and more

Transformer models are the de-facto standard in modern NLP. They have proven themselves as the most expressive, powerful models for language by a large margin, beating all major language-based benchmarks time and time again.

In this course, we cover everything you need to get started with building cutting-edge performance NLP applications using transformer models like Google AI’s BERT, or Facebook AI’s DPR.

We cover several key NLP frameworks including:

  • HuggingFace’s Transformers
  • TensorFlow 2
  • PyTorch
  • spaCy
  • NLTK
  • Flair

And learn how to apply transformers to some of the most popular NLP use-cases:

  • Language classification/sentiment analysis
  • Named entity recognition (NER)
  • Question and Answering
  • Similarity/comparative learning

Throughout each of these use-cases we work through a variety of examples to ensure that what, how, and why transformers are so important. Alongside these sections we also work through two full-size NLP projects, one for sentiment analysis of financial Reddit data, and another covering a fully-fledged open domain question-answering application.

All of this is supported by several other sections that encourage us to learn how to better design, implement, and measure the performance of our models, such as:

  • History of NLP and where transformers come from
  • Common preprocessing techniques for NLP
  • The theory behind transformers
  • How to fine-tune transformers

We cover all this and more, I look forward to seeing you in the course!

What you’ll learn

  • Industry standard NLP using transformer models
  • Build full-stack question-answering transformer models
  • Perform sentiment analysis with transformers models in PyTorch and TensorFlow
  • Advanced search technologies like Elasticsearch and Facebook AI Similarity Search (FAISS)
  • Create fine-tuned transformers models for specialized use-cases
  • Measure performance of language models using advanced metrics like ROUGE
  • Vector building techniques like BM25 or dense passage retrievers (DPR)
  • An overview of recent developments in NLP
  • Understand attention and other key components of transformers
  • Learn about key transformers models such as BERT
  • Preprocess text data for NLP
  • Named entity recognition (NER) using spaCy and transformers
  • Fine-tune language classification models
Table of Contents

Introduction
1 Introduction
2 Course Overview
3 Environment Setup
4 CUDA Setup
5 Alternative Colab Setup
6 Alternative Local Setup

NLP and Transformers
7 The Three Eras of AI
8 Pros and Cons of Neural AI
9 Word Vectors
10 Recurrent Neural Networks
11 Long Short-Term Memory
12 Encoder-Decoder Attention
13 Self-Attention
14 Multi-head Attention
15 Positional Encoding
16 Transformer Heads

Preprocessing for NLP
17 Stopwords
18 Tokens Introduction
19 Model-Specific Special Tokens
20 Stemming
21 Lemmatization
22 Unicode Normalization – Canonical and Compatibility Equivalence
23 Unicode Normalization – Composition and Decomposition
24 Unicode Normalization – NFD and NFC
25 Unicode Normalization – NFKD and NFKC

Attention
26 Attention Introduction
27 Alignment With Dot-Product
28 Dot-Product Attention
29 Self Attention
30 Bidirectional Attention
31 Multi-head and Scaled Dot-Product Attention

Language Classification
32 Introduction to Sentiment Analysis
33 Prebuilt Flair Models
34 Introduction to Sentiment Models With Transformers
35 Tokenization And Special Tokens For BERT
36 Making Predictions

[Project] Sentiment Model With TensorFlow and Transformers
37 Project Overview
38 Getting the Data (Kaggle API)
39 Preprocessing
40 Building a Dataset
41 Dataset Shuffle, Batch, Split, and Save
42 Build and Save
43 Loading and Prediction

Long Text Classification With BERT
44 Classification of Long Text Using Windows
45 Window Method in PyTorch

Named Entity Recognition (NER)
46 Introduction to spaCy
47 Extracting Entities
48 Authenticating With The Reddit API
49 Pulling Data With The Reddit API
50 Extracting ORGs From Reddit Data
51 Getting Entity Frequency
52 Entity Blacklist
53 NER With Sentiment
54 NER With roBERTa

Question and Answering
55 Open Domain and Reading Comprehension
56 Retrievers, Readers, and Generators
57 Intro to SQuAD 2.0
58 Processing SQuAD Training Data
59 (Optional) Processing SQuAD Training Data with Match-Case
60 Our First Q&A Model

Metrics For Language
61 Q&A Performance With Exact Match (EM)
62 ROUGE in Python
63 Applying ROUGE to Q&A
64 Recall, Precision and F1
65 Longest Common Subsequence (LCS)
66 Q&A Performance With ROUGE

Reader-Retriever QA With Haystack
67 Intro to Retriever-Reader and Haystack
68 What is Elasticsearch
69 Elasticsearch Setup (Windows)
70 Elasticsearch Setup (Linux)
71 Elasticsearch in Haystack
72 Sparse Retrievers
73 Cleaning the Index
74 Implementing a BM25 Retriever
75 What is FAISS
76 FAISS in Haystack
77 What is DPR
78 The DPR Architecture
79 Retriever-Reader Stack
80 Further Materials for Faiss

[Project] Open-Domain QA
81 ODQA Stack Structure
82 Creating the Database
83 Building the Haystack Pipeline

Similarity
84 Introduction to Similarity
85 Extracting The Last Hidden State Tensor
86 Sentence Vectors With Mean Pooling
87 Using Cosine Similarity
88 Similarity With Sentence-Transformers
89 Further Learning

Pre-Training Transformer Models
90 Visual Guide to BERT Pretraining
91 Introduction to BERT For Pretraining Code
92 BERT Pretraining – Masked-Language Modeling (MLM)
93 BERT Pretraining – Next Sentence Prediction (NSP)
94 The Logic of MLM
95 Pre-training with MLM – Data Preparation
96 Pre-training with MLM – Training
97 Pre-training with MLM – Training with Trainer
98 The Logic of NSP
99 Pre-training with NSP – Data Preparation
100 Pre-training with NSP – DataLoader
101 The Logic of MLM and NSP
102 Pre-training with MLM and NSP – Data Preparation

Homepage