English | MP4 | AVC 1280×720 | AAC 44KHz 2ch | 102 lectures (8h 26m) | 2.47 GB
Build Artificial Intelligence (AI) agents using Deep Reinforcement Learning and PyTorch: From basic DQN to Rainbow DQN
This is the most complete Advanced Reinforcement Learning course on Udemy. In it, you will learn to implement some of the most powerful Deep Reinforcement Learning algorithms in Python using PyTorch and PyTorch lightning. You will implement from scratch adaptive algorithms that solve control tasks based on experience. You will learn to combine these techniques with Neural Networks and Deep Learning methods to create adaptive Artificial Intelligence agents capable of solving decision-making tasks.
This course will introduce you to the state of the art in Reinforcement Learning techniques. It will also prepare you for the next courses in this series, where we will explore other advanced methods that excel in other types of task.
The course is focused on developing practical skills. Therefore, after learning the most important concepts of each family of methods, we will implement one or more of their algorithms in jupyter notebooks, from scratch.
Leveling modules:
- Refresher: The Markov decision process (MDP).
- Refresher: Q-Learning.
- Refresher: Brief introduction to Neural Networks.
- Refresher: Deep Q-Learning.
Advanced Reinforcement Learning:
- PyTorch Lightning.
- Hyperparameter tuning with Optuna.
- Reinforcement Learning with image inputs
- Double Deep Q-Learning
- Dueling Deep Q-Networks
- Prioritized Experience Replay (PER)
- Distributional Deep Q-Networks
- Noisy Deep Q-Networks
- N-step Deep Q-Learning
- Rainbow Deep Q-Learning
What you’ll learn
- Master some of the most advanced Reinforcement Learning algorithms.
- Learn how to create AIs that can act in a complex environment to achieve their goals.
- Create from scratch advanced Reinforcement Learning agents using Python’s most popular tools (PyTorch Lightning, OpenAI gym, Optuna)
- Learn how to perform hyperparameter tuning (Choosing the best experimental conditions for our AI to learn)
- Fundamentally understand the learning process for each algorithm.
- Debug and extend the algorithms presented.
- Understand and implement new algorithms from research papers.
Table of Contents
Introduction
1 Introduction
2 Reinforcement Learning series
3 Google Colab
4 Where to begin
5 Complete code
Refresher The Markov Decision Process (MDP)
6 Module overview
7 Elements common to all control tasks
8 The Markov decision process (MDP)
9 Types of Markov decision process
10 Trajectory vs episode
11 Reward vs Return
12 Discount factor
13 Policy
14 State values v(s) and action values q(s,a)
15 Bellman equations
16 Solving a Markov decision process
Refresher Q-Learning
17 Module overview
18 Temporal difference methods
19 Solving control tasks with temporal difference method
20 Q-Learning
21 Advantages of temporal difference methods
Refresher Brief introduction to Neural Networks
22 Module overview
23 Function approximators
24 Artificial Neural Networks
25 Artificial Neurons
26 How to represent a Neural Network
27 Stochastic Gradient Descent
28 Neural Network optimization
Refresher Deep Q-Learning
29 Module overview
30 Deep Q-Learning
31 Experience replay
32 Target Network
PyTorch Lightning
33 PyTorch Lightning
34 Link to the code notebook
35 Introduction to PyTorch Lightning
36 Create the Deep Q-Network
37 Create the policy
38 Create the replay buffer
39 Create the environment
40 Define the class for the Deep Q-Learning algorithm
41 Define the play episode() function
42 Prepare the data loader and the optimizer
43 Define the train step() method
44 Define the train epoch end() method
45 [Important] Lecture correction
46 Train the Deep Q-Learning algorithm
47 Explore the resulting agent
Hyperparameter tuning with Optuna
48 Hyperparameter tuning with Optuna
49 Link to the code notebook
50 Log average return
51 Define the objective function
52 Create and launch the hyperparameter tuning job
53 Explore the best trial
Double Deep Q-Learning
54 Maximization bias and Double Deep Q-Learning
55 Link to the code notebook
56 Create the Double Deep Q-Learning algorithm
57 Check the resulting agent
Dueling Deep Q-Networks
58 Dueling Deep Q-Networks
59 Link to the code notebook
60 Create the dueling DQN
61 Observation and reward normalization
62 Create the environment – Part 1
63 Create the environment – Part 2
64 Implement Deep Q-Learning
65 Check the resulting agent
Prioritized Experience Replay
66 Prioritized Experience Replay
67 Link to the code notebook
68 DQN for visual inputs
69 Prioritized Experience Repay Buffer
70 Create the environment
71 Implement the Deep Q-Learning algorithm with Prioritized Experience Replay
72 Errata Lecture 70
73 Launch the training process
74 Check the resulting agent
Noisy Deep Q-Networks
75 Noisy Deep Q-Networks
76 Link to the code notebook
77 Create the noisy linear layer class
78 Create the Deep Q-Network
79 Create the policy
80 Create the environment
81 Train the algorithm
82 Check the results
N-step Deep Q-Learning
83 N-step Deep Q-Learning
84 Link to the code notebook
85 N-step Deep Q-Learning – Part 1
86 N-step Deep Q-Learning – Part 2
87 N-step Deep Q-Learning – Part 3
88 Check results
Distributional Deep Q-Networks
89 Distributional Deep Q-Networks
90 Link to the code notebook
91 Create the distributional DQN – Part 1
92 Create the distributional DQN – Part 2
93 Create the policy
94 Create the environment
95 Adapt the algorithm Constructor and sample function
96 Adapt the algorithm Training step – Part 1
97 Adapt the algorithm Training step – Part 2
98 Adapt the algorithm Training step – Part 3
99 Adapt the algorithm Training step – Part 4
100 Launch the training process
Final steps
101 Next steps
102 Next steps
Resolve the captcha to access the links!