Large language models (LLMs) have garnered widespread social attention due to their generalization ability and strong performance across various tasks. In this course, we will introduce the development history, key technologies, and existing challenges in the field of artificial intelligence (AI), with a special focus on LLMs. We will discuss the potential of progressing towards artificial general intelligence, helping students to engage with the wave of technological changes driven by AI.
Introduction to Large Language Models
Instructors
${ item.name }$
Teaching Assistants
${ item.name }$
Speakers
${ item.name }$
Logistics
- ${ item }$
Content
Prerequisites
- Python: All course assignments and the final project need to be completed using Python. Therefore, students should be familiar with Python.
- Basic Linear Algebra: When introducing the basic principles of neural networks and LLMs, we will involve matrix operations. Thus, students should be familiar with matrix notation and operations.
- For non-computer science students: If your goal is to grasp the fundamental knowledge and concepts related to LLMs without delving into code implementation, this course remains accessible.
Lectures
${ scope.row.Materials }$
- ${item}$ ${item}$
${
scope.row.SuggestedCodeRepo }$
- Torch Tutorials https://pytorch.org/tutorials/ ${item}$ ${item}$
Coursework
Assignments
Large language models (LLMs) have garnered widespread social attention
due to their generalization ability and strong performance across
various tasks. In this course, we will introduce the development
history, key technologies, and existing challenges in the field of
artificial intelligence (AI), with a special focus on LLMs. We will
discuss the potential of progressing towards artificial general
intelligence, helping students to engage with the wave of technological
changes driven by AI.
- Assignment 1: Inference and Evaluation - Utilize LLMs to make predictions and evaluate their performance.
- Assignment 2: Finetuning - Adapt LLMs to specific tasks.
- Assignment 3: Software Development Agents - Create software development agents driven by LLMs.
Final Project
The final project encourages students to apply the knowledge of LLMs in a practical
setting. Students are required to independently select a topic and develop an open-source project that
leverages LLMs to solve problems relevant to their daily life or work.
- Team Size: Each team may consist of 1-2 members.
- Project Topic: Students can select any topic related to LLMs. In the second week of the course (July 8, morning), each team needs to present their project topics. The instructor and TAs will provide feedback to each team.
- Computing Resources: Each team will be provided with RTX 4090/3090 GPUs for model training. Students can also access other cloud computing resources, such as Google Codalab, by yourself.
- Presentation: Teams will present their projects using posters on July 19. The teaching team will grade each project based on the workload and practicality.
