Summary

This course explores the foundations and frontiers of web-scale information systems, from classical retrieval techniques to modern LLM-powered search. We study topics including: web crawling, deep/dark web discovery, classical and neural information retrieval (BM25, ColBERT, DPR), large language models for search (RAG), knowledge graphs, social network analysis, recommendation systems, and computational advertising. The course involves presentations and study of research papers as well as hands-on projects.

Course Information

  • Spring semester 2026
  • Class: Wed 10:00-13:00
  • Instructor: Alexandros Ntoulas
    antoulas@di.uoa.gr

Announcements

  • __/__: Welcome to M151! Course materials will be posted throughout the semester.

References

The course material comes primarily from research papers from conferences such as WWW, WSDM, SIGIR, KDD, NeurIPS, EMNLP, ICLR, etc. The papers for each topic are listed in the schedule table below.

For additional background, you may also consult the following books (not required):

Syllabus & Schedule

Lecture Topic Handouts/Assignments
MODULE 1: Web Foundations & Discovery
Lecture 1 Web Architecture & Modern Crawling
  • IRLbot: Scaling to 6 Billion Pages and Beyond, H. Lee et al., WWW 2008 [pdf]
  • Efficient Crawling Through URL Ordering, J. Cho et al., WWW 1998 [pdf]
Lecture 2 Hidden Web, Dark Web &
Specialized Discovery
  • Google's Deep-Web Crawl, J. Madhavan et al., VLDB 2008 [pdf]
  • CRATOR: A Dark Web Crawler, D. De Pascale et al., arXiv 2024 [pdf]
MODULE 2: Information Retrieval - Classical to Neural
Lecture 3 Classical Information Retrieval & Indexing
  • Introduction to Information Retrieval (Ch. 1-2, 6), C. Manning et al. [online]
  • A Study of Smoothing Methods for Language Models Applied to IR, C. Zhai, J. Lafferty, SIGIR 2001 [pdf]
Lecture 4 Neural Information Retrieval
  • ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT, O. Khattab, M. Zaharia, SIGIR 2020 [pdf]
  • Dense Passage Retrieval for Open-Domain Question Answering, V. Karpukhin et al., EMNLP 2020 [pdf]
Lecture 5 Scalable Search Systems & Infrastructure
  • Efficient Query Processing for Scalable Web Search, A. Broder et al., Communications of the ACM [pdf]
  • Billion-Scale Similarity Search with GPUs (FAISS), J. Johnson et al., IEEE Trans. Big Data 2019 [pdf]
MODULE 3: LLMs Transform Search
Lecture 6 Retrieval-Augmented Generation (RAG)
  • Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, P. Lewis et al., NeurIPS 2020 [pdf]
  • Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection, A. Asai et al., ICLR 2024 [pdf]
Lecture 7 LLMs as Search Engines &
Conversational IR
  • Query Rewriting for Retrieval-Augmented Large Language Models, X. Ma et al., EMNLP 2023 [pdf]
  • Evaluating Verifiability in Generative Search Engines, N. Liu et al., EMNLP 2023 [pdf]
MODULE 4: Knowledge, Networks & Applications
Lecture 8 Knowledge Graphs &
Entity-Centric Search
  • Unifying Large Language Models and Knowledge Graphs: A Roadmap, S. Pan et al., IEEE TKDE 2024 [pdf]
  • From Local to Global: A Graph RAG Approach to Query-Focused Summarization, D. Edge et al. (Microsoft), 2024 [pdf]
Lecture 9 Social Networks: Structure,
Influence & Misinformation
  • Maximizing the Spread of Influence through a Social Network, D. Kempe et al., KDD 2003 [pdf]
  • Combating Misinformation in the Age of LLMs: Opportunities and Challenges, C. Chen et al., AI Magazine 2024 [pdf]
Lecture 10 Recommendation Systems &
Computational Advertising
  • Neural Collaborative Filtering, X. He et al., WWW 2017 [pdf]
  • Practical Lessons from Predicting Clicks on Ads at Facebook, X. He et al., ADKDD 2014 [pdf]
Paper Presentations Project presentations TBD
Paper Presentations Project presentations TBD
Final Exam Final exam TBD