Date/Time
Date(s) - 05/27/2025 - 05/31/2030
12:00 AM
Add to Google Calendar or iCal/Outlook Calendar
Instructor
Dr. Li-C Wang is a professor at ECE department at the University of CA, Santa Barbara.
Learning Objectives
Large Language Models (LLMs) are a class of machine learning models that have revolutionized the field of Natural Language Processing (NLP). LLMs are capable of understanding human-like text in a wide range of applications, including text classification, translation, and even intelligent chatbots. This course provides an introduction to the Transformer architecture, which is the foundation of many state-of-the-art LLMs, originating in the models BERT and GPT. The course covers the development of language models, both in the general capabilities sense and in the specific areas of training and construction. A set of homework assignments are included to give experience with the fundamental training and use of LLMs, both in the development-side context and the foundation model view.
This course introduces the concept of the language model, a machine learning model that can understand (via classification, translation, or emergent few-shot and zero-shot in-context learning) tasks on natural language text data. The course begins with a brief summary of the Recurrent Neural Network (RNN) and Long-Short-Term-Memory (LSTM) architectures, as background for a greater discussion of the Transformer architecture. We identify the Transformer’s place as the foundation of many state-of-the-art Large Language Models (LLMs), including BERT and GPT, and discuss the development of language models in the general capabilities (use in practice) sense and in the specific areas of training and construction. The course is divided into three modules:
- Unit 1: Introduction to Large Language Models (LLMs): Jumps into the Transformer architecture, via brief summary of the Recurrent Neural Network (RNN) and Long-Short-Term-Memory (LSTM) architectures. Uses this as a springboard to introduce the development of language models and their replacement of the NLP pipeline with models showcasing emergent capabilities.
- Unit 2: Introduction to IEA-Plot – Building an AI Assistant in Practice: Showcases a real research tool demonstrating application of modern large language models to silicon test data analytics tasks without sacrificing trust and reliability of displayed results. Shows the internal “thought process” of the tool which maintains reliability, and how that behavior was built from less reliable and domain-specific tools like chat-capable language models.
- Unit 3: Origin of Language Models – Self-Supervised Learning: Thoroughly explains the training methods used to create BERT and GPT large language models and their most effective predecessors. Shows how these models process the meaning of their inputs internally, and where their understanding of that meaning is developed.
Prerequisites:
- An understanding of the high-level programming language Python and basic mathematics (algebra, linear algebra, trigonometry, derivatives) is assumed.
- Knowledge of the fundamental units of microelectronics fabrication (wafers, die, lithography) is helpful but not required.
- MEST Micro-Certificate: Introduction to Machine Learning is highly recommended.
Biography
Li-C. Wang is a professor at ECE department at the University of CA, Santa Barbara. He received PhD in 1996 from the University of Texas, Austin, and was previously with Motorola PowerPC Design Center. Starting from 2003, his research has focused on investigating how machine learning could be utilized in design and test flows, where he had published more than 100 papers and supervised 22 PhD theses on related subjects. Before that, his research spanned across multiple topics in EDA and test, including microprocessor test and verification, statistical timing analysis, defect-oriented testing, and SAT solvers. He received 10 Best Paper Awards and 2 Honorable Mentioned Paper Awards from major conferences, including recent best paper awards from ITC 2022, ITC 2020, VTS 2016, and VLSI-DAT 2019. He is the recipient of the 2010 Technical Excellence Award from Semiconductor Research Corporation (SRC) for his research contributions in data mining for test and validation. He is the recipient of the 2017 IEEE-TTTC Bob Madge Innovation Award. He is an IEEE fellow and served as the General Chair of the International Test Conference (ITC) in 2017, 2018, and 2023.
