1 Book Reinforcement Learning: Algorithms and Convergence

1.1 Artificial Intelligence and Reinforcement Learning

Reinforcement learning is the field of artificial intelligence (AI) and machine learning that has been crucial for the progress in artificial intelligence in the past few years. Playing chess, Go, computer games such as Atari 2600 games and Gran Turismo, and card games such as poker and Schnapsen at superhuman levels has all been made possible by reinforcement learning. The final training step of AI systems such as ChatGPT is reinforcement learning. Most major websites use reinforcement learning.

In this sense, we are all users of reinforcement learning. Despite the great progress, there are still many challenges and open questions.

Do you want to know how these modern algorithms work? My book Reinforcement Learning: Algorithms and Convergence discusses the most important learning algorithms and their convergence theory as well as recent and advanced topics such as deep reinforcement learning, distributional reinforcement learning, and large language models.

1.2 Table of Contents

Introduction
Markov Decision Processes and Dynamic Programming
Monte-Carlo Methods and Theory
Temporal-Difference Learning
Q-Learning and Theory
On-Policy Prediction with Approximation
Policy-Gradient Methods
Hamilton-Jacobi-Bellman Equations
Deep Reinforcement Learning
Distributional Reinforcement Learning
Large Language Models
Appendices:
- Measure and Probability Theory
- Software Libraries
Bibliography
Index

1.3 Availability

Currently, lecture notes are available to the students in my classes (softcover or PDF). If you are also interested, please send me an email.

1.4 Feedback

Please send me an email with any feedback.