Machine Learning: A Bayesian and Optimization Perspective

S. Theodoridis
Academic Press, 2015 (1050 pages).
ISBN: 9780128015223


This tutorial text gives a unifying perspective on machine learning by covering both probabilistic and deterministic approaches -which are based on optimization techniques – together with the Bayesian inference approach, whose essence lies in the use of a hierarchy of probabilistic models.
The book presents the major machine learning methods as they have been developed in different disciplines, such as statistics, statistical and adaptive signal processing and computer science. Focusing on the physical reasoning behind the mathematics, all the various methods and techniques are explained in depth, supported by examples and problems, giving an invaluable resource to the student and researcher for understanding and applying machine learning concepts.
The book builds carefully from the basic classical methods to the most recent trends, with chapters written to be as self-contained as possible, making the text suitable for different courses: pattern recognition, statistical/adaptive signal processing, statistical/Bayesian learning, as well as short courses on sparse modeling, deep learning, and probabilistic graphical models.

Pattern Recognition

S. Theodoridis, K. Koutroumbas
Academic Press, 1st Edition 1998, 2nd Edition 2003, 3nd Edition 2006, 4th Edition 2009. 960 pages.
ISBN: 9781597492720


The book has been among the best selling titles of Academic Press and it has also been translated into Chinese and into Greek (more than 25000 copies sold).
In this book, Supervised, Semisupervised and Unsupervised (Clustering) are treated in an equally balanced way.


show/hide reviews

"I cut my pattern recognition teeth on a draft version of Duda and Hart (1973). Over subsequent decades, I consistently did two things: (i) recommended Duda and Hart as the best book available on pattern recognition; and (ii) wanted to write the next best book on this topic.
I stopped (i) when the first edition of S. Theodoridis and K. Koutroumbas' book appeared, and it supplanted the need for (ii).
It was, and is, the best book that has been written on the subject since Duda and Hart's seminal original text. Buy it - you'll be happy you did."

Jim Bezdek, University of West Florida and Senior Fellow, U. of Melbourne (Australia).


"I consider the fourth edition of the book Pattern Recognition, by S. Theodoridis and K. Koutroumbas as the "Bible of Pattern Recognition"

Simon Haykin, McMaster University, Canada.


"I have taught a graduate course on statistical pattern recognition for more than twenty five years during which I have used many books with different levels of satisfaction. Recently, I adopted the book by Theodoridis and Koutroumbas (4th edition) for my graduate course on statistical pattern recognition at University of Maryland. This course is taken by students from electrical engineering, computer science, linguistics and applied mathematics. The comprehensive book by Thedoridis and Koutroumbas covers both traditional and modern topics in statistical pattern recognition in a lucid manner, without compromising rigor. This book elegantly addresses the needs of graduate students from the different disciplines mentioned above. This is the only book that does justice to both supervised and unsupervised (clustering) techniques. Every student, researcher and instructor who is interested in any and all aspects of statistical pattern recognition will find this book extremely satisfying. I recommend it very highly."

Rama Chellappa, University of Maryland.


"The book Pattern Recognition, by Profs. Sergios Theodoridis and Konstantinos Koutroumbas, has rapidly become the "bible" for teaching and learning the ins and outs of pattern recognition technology. In my own teaching, I have utilized the material in the first four chapters of the book (from basics to Bayes Decision Theory to Linear Classifiers and finally to Nonlinear Classifiers) in my class on fundamentals of speech recognition and have found the material to be presented in a clear and easily understandable manner, with excellent problems and ideas for projects. My students have all learned the basics of pattern recognition from this book and I highly recommend it to any serious student in this area."

Prof. Lawrence Rabiner


"This book is an excellent reference for pattern recognition, machine learning, and data mining. It focuses on the problems of classification and clustering, the two most important general problems in these areas. This book has tremendous breadth and depth in its coverage of these topics; it is clearly the best book available on the topic today. The new edition is an excellent up-to-date revision of the book. I have especially enjoyed the new coverage provided in several topics, including new viewpoints on Support Vector Machines, and the complete in-depth coverage of new clustering methods. This is a standout characteristic of this book: the coverage of the topics is solid, deep, and principled throughout. The book is very successful in bringing out the important points in each technique, while containing lots of interesting examples to explain complicated concepts. I believe the section on dimensionality reduction is an excellent exposition on this topic, among the best available, and this is just one example. Combined with a coverage unique in its extend, this makes the book appropriate for use as a reference, as a textbook for upper level undergraduate or graduate classes, and for the practitioner that wants to apply these techniques in practice. I am a professor in Computer Science. Although pattern recognition is not my main focus, I work in the related fields of data mining and databases. I have used this book for my own research and, very successfully, as teaching material. I would strongly recommend this book to both the academic student and the professional."

Dimitrios Gunopoulos, University of California, Riverside, USA hide

Academic Press Library in Signal Processing

R. Chellapa, S. Theodoridis (Eds)
Vols 1-4, Academic Press, 2014. ISBN: 9780124166165


This is a four volume book presenting all major directions in Signal Processing, starting from classical theory till cutting edge research topics. All chapters have been contributed by well known world experts.

Introduction to Pattern Recognition: A MATLAB based approach

S. Theodoridis, A. Pikrakis, K. Koutroumbas, D. Cavouras
Academic Press, 2010. ISBN: 9780123744869


This book is intended for the new-comer in the field. It combines a non-mathematical summary of the main concepts and it focuses on MatLab-based examples. The book has been translated into Korean and Greek.

Adaptive System Identification and Signal Processing Algorithms

N. Kalouptsidis, S. Theodoridis (eds.)
Prentice Hall 1993.
Back to top