This is a data systems course: how large-scale data is stored, organized, processed, streamed, and fed to machine learning and LLMs. We study the designs behind the systems that run the modern data platform — MapReduce and its successors, distributed file systems and object storage, columnar formats and the lakehouse, key-value stores and LSM trees, Spark and vectorized analytical engines, Kafka and stream processing, sketches and sampling, graph processing, the data pipelines behind large language models, and large-scale experimentation. The course is built around research papers (two per lecture), hands-on labs on your own laptop, a paper presentation, and a written final exam. The recurring theme is not reading everything: the best data systems are the ones that touch the least data to answer a question.

The course material comes primarily from research papers (OSDI, SOSP, SIGMOD, VLDB, NSDI, KDD, etc.), listed per lecture in the schedule below. No textbook is required. For background:

• Designing Data-Intensive Applications, Martin Kleppmann, O’Reilly, 2017.
• Database Internals, Alex Petrov, O’Reilly, 2019.
• CMU 15-445/645 Database Systems (lecture videos) for a refresher on DBMS fundamentals.