Welcome to the Artificial Intelligence II course

The course concentrates on the study of modern deep learning techniques and their use in natural language processing.

Prerequisites

• Artificial Intelligence (Semester 5)

Topics

The course covers a wide range of AI topics and their use in NLP: introduction to machine learning, regression, perceptron, neural networks, backpropagation, deep neural network training, word vectors, word2vec and related models, language modeling and RNNs, vanishing gradients, LSTMs/GRUs, machine translation, seq2seq and attention, transformers, large language models (BERT, GPT family, GEMINI family etc.).

Study material

ML Books

• Machine Learning: From the Classics to Deep Networks, Transformers, and Diffusion Models
• Deep Learning: Foundations and Concepts
• Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow, 2nd Edition by Aurélien Géron
• Dive into Deep Learning
• Introduction to Deep Learning by Eugene Charniak
• Deep Learning - An MIT Press book by Ian Goodfellow and Yoshua Bengio and Aaron Courville
• Pattern Recognition and Machine Learning by Christopher M. Bishop
• Mathematics for Machine Learning
• Approaching (Almost) Any Machine Learning Problem
• The Principles of Deep Learning Theory
• Introduction to Probability for Data Science
• Patterns, predictions, and actions: A story about machine learning
• Foundations of Data Science
• Big Data and AI Strategies, Machine Learning and Alternative Data Approach to Investing
• Understanding Deep Learning by Simon J.D. Prince
• A Cookbook of Self-Supervised Learning
• A Little Book of Deep Learning
• Linear Algebra by Gilbert Strang
• Linear Algebra and Learning from Data (2019)

NLP Books

• Speech and Language Processing (3rd ed. draft) Dan Jurafsky and James H. Martin
• A Primer on Neural Network Models for Natural Language Processing by Yoav Goldberg
• Build a large language model from scratch by Sebastian Raschka
• Natural Language Processing by Jacob Eisenstein
• Natural Language Processing with PyTorch by Delip Rao, Brian McMahan
• The Hundred-Page Language Models Book
• Deep Learning by Ian Goodfellow, Yoshua Bengio and Aaron Courville
• Natural Language Processing with Transformers by Lewis Tunstall, Leandro von Werra and Thomas Wolf

LLM Books

• The Hundred-Page Language Models Book: hands-on with PyTorch
• Hands-On Large Language Models: Language Understanding and Generation
• Build a Large Language Model from Scratch

Related Courses

• Natural Language Processing with Deep Learning (Stanford)
• Neural Networks for NLP (CMU CS 11-747)
• Introduction to Machine Learning Lecture Notes for COS 324 at Princeton University

Articles

• The Matrix Calculus You Need For Deep Learning
• A Machine Learning Primer by Mihail Eric

Web sites with code

• https://www.fast.ai
• https://huggingface.co
• Deep explanations of machine learning and related topics, by Terence Parr
• Seeing Theory: make statistics more accessible through interactive visualizations